Capturing Clinical Knowledge: The Digital Modeling of Expert Knowledge Concerning the Care for Patients with Chronic Obstructive Pulmonary Disease

Capturing Clinical Knowledge: The Digital Modeling of Expert Knowledge Concerning the Care for Patients with Chronic Obstructive Pulmonary Disease | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Capturing Clinical Knowledge: The Digital Modeling of Expert Knowledge Concerning the Care for Patients with Chronic Obstructive Pulmonary Disease Eric Edelman, Esmee Bellemakers, Fabian Tijssen, Popke Rein Munniksma, and 5 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6819489/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 11 You are reading this latest preprint version Abstract Background Encoding medical knowledge in a digital clinical knowledge model (CKM) enables its usage for automation and decision support. Formalized sources of knowledge are usually not sufficient to construct a complete model. Clinical experts may also hold relevant, implicit knowledge that they have gained through experience or from other unrecorded sources. Our aim was to study whether early modeling of medical knowledge in a CKM and the fictional application of the resulting model together with clinical experts might help elicit this semi-hidden, but highly relevant domain knowledge. Methods We created a CKM to support patients suffering from chronic obstructive pulmonary disease in self-managing their condition by generating recommendations based on measurements and questionnaires they perform at home. We subsequently interviewed 8 pulmonary experts about their recommendations for synthetic patient cases versus those generated by the CKM. At the same time, we collected feedback from the professionals to study their attitude towards the CKM and its generated recommendations. Results The interviews enabled us to elicit further domain knowledge on various themes: retaking measurements, asking the patient additional questions, contacting the care professional, medication, continuation of monitoring, and lifestyle advice. Secondly, the elicited knowledge revealed interprofessional differences between different types of care professionals and within groups of the same type. Additionally, our results show a trend that the experienced professionals accepted the model’s advice more readily than other groups. Conclusions The themes we identified indicate that case-based interviewing is a suitable technique for knowledge elicitation regarding clinical knowledge. The interprofessional differences in recommendations form a hurdle in expanding the accepted knowledge encoded in the CKM. The experienced professionals being more accepting of the model’s advice contrasts with existing literature. This highlights the need for further research to understand the correlation between a care professional’s experience and the adoption of automatically generated recommendations. Patients were intentionally excluded from this preliminary evaluation of the CKM to first determine if the model aligned with current medical practice. Future studies should include both patients and care professionals to assess the tool’s usability. clinical knowledge model knowledge engineering chronic obstructive pulmonary disease remote patient monitoring automation decision support Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Background Automation and decision support in healthcare may be enabled by encoding medical knowledge in digital clinical knowledge models (CKMs).( 1 ) Clinical knowledge modeling consists of two parallel activities: one is determining which decisions are made in the domain being modeled and which information is required to make them, and the other constitutes the conceptual modeling of the domain knowledge (i.e. how the information leads to the ability to make decisions). Examples of such decisions are whether a patient requires surgery, whether they are eligible for discharge, whether they are at risk of osteoporosis, which medication to prescribe, what triage category they should be assigned, etc. While making single decisions in isolation is straightforward, healthcare often requires a multitude of interrelated decisions to be made at the same time, creating a complex network of required information and rules on its processing. Klemann et al. have described the crucial role of information processing in creating a robust and resilient healthcare system.( 2 , 3 ) Following their reference to Galbraith’s information processing view on organizational design, a CKM greatly increases the information processing capabilities of a healthcare system. It does this by making the network of rule-based clinical decisions manageable and applicable. Once provided with input, all modeled decisions are continuously executed by the CKM. The final output is generated based on the collected outcomes of those decisions. The model’s output can therefore consist of multiple aspects, such as a request for the patient to fill in another questionnaire, as well as treatment recommendations for the care professional, such as the suggestion to increase a medication dose. Finding the decisions and required information relevant to a CKM generally starts by analyzing existing protocols, guidelines, and any other formalized sources of knowledge on the topic at hand. However, this is usually not sufficient to construct a complete model. Clinical experts may also hold relevant, implicit knowledge that they have gained through experience or from other unrecorded sources.( 1 , 4 – 6 ) The aim of the research described in this paper was to study whether early modeling of medical knowledge in a CKM and the fictional application of the resulting model together with clinical experts might enable us to elicit this semi-hidden, but highly relevant domain knowledge. Secondly, we looked for potential interprofessional differences in treatment strategies. Basing our definition on De Gans et al., interprofessional differences are differences among healthcare professionals from different professions, but within the same clinical specialty.( 7 ) Investigating the presence of interprofessional clinical differences yields insights into the consistency and quality of the interpretation and execution of current clinical protocols. Any disagreements may require revisions of protocols or agreements amongst clinicians to restore consistency, which is essential for producing a CKM that is trusted by all. Finally, we collected feedback from the professionals we included on further cases to study their attitude towards the CKM and its generated recommendations. Methods We examined an example of clinical knowledge modeling that took place at the Department of Respiratory Medicine of the Maastricht University Medical Center+ (MUMC+). The resultant CKM is to support patients suffering from chronic obstructive pulmonary disease (COPD) in self-managing their chronic condition, thereby relieving pulmonary specialists from some of the monitoring and treatment of these patients. In a series of interviews based on synthetic patient cases, we gathered data from participating clinical experts. For cases 1 –3, this concerned their clinical recommendations for the same cases. This allowed us to make a comparison with the CKM-generated recommendations, providing insights into 1. the quantity and quality of additional domain knowledge this method provided and 2. the level of interprofessional variation in clinical recommendations. Additionally, we calculated the adoption rate by the experts of the CKM-generated recommendations for cases 4 –6, and coded the recurring themes in the experts’ responses to these recommendations. This resulted in insights into the experts’ attitude towards the CKM and its recommendations. Figure 1 displays an overview of the study performed. Further details on the three major components follow below. CKM The CKM was created in collaboration with the Department of Respiratory Medicine of the MUMC+. The experts that contributed to the model were not involved in the current study. Daily inputs were three vital parameters (blood oxygen saturation, heart rate, and respiratory rate) and the COPD Exacerbation Recognition Tool (CERT), which aims to increase a patient’s ability to report on their exacerbations.( 8 ) Patients could be requested to complete two additional questionnaires when needed: the Exacerbations of Chronic Pulmonary Disease Tool (EXACT) and a custom questionnaire. EXACT is a patient-reported outcome measure for evaluating the frequency, severity, and duration of COPD exacerbations.( 9 ) It expands on topics also covered in the CERT questionnaire, such as increased intensity of coughing, increased amount of phlegm, shortness of breath, difficulty breathing, and impairment of performing activity. The model triggered EXACT once a patient’s symptoms exceeded a certain level of severity detected by the CERT. The second optional questionnaire was drafted by the experts who contributed to the model. This acute problem questionnaire posed questions about pain when breathing, body temperature, coughing up blood, palpitations, shortness of breath when lying down, and swollen ankles. These inputs were added to enable compliance with the Dutch national guidelines on the treatment of COPD. All these inputs jointly fed the decision rules included in the CKM, which were continuously executed by the application. After each evaluation, the model generated output based on the results. Figure 2 summarizes the functionality of the model. Synthetic patient cases We generated synthetic data for six patient cases that would be evaluated by both the CKM and our included healthcare professionals. Synthetic data has been described as fake data that faithfully represents the real-world data that is taken as its reference, making it fictional but realistic.( 10 , 11 ) We followed the method described by Murtaza and colleagues.( 12 ) They distinguish between data-driven, knowledge-driven, and hybrid approaches. In our case, we employed their knowledge-driven approach for the possibility to tailor the patient cases to specific scenarios without needing to look for real patients that matched the criteria. Collaboration with an experienced pulmonologist (who was not further involved with the study) across three in-person sessions ensured the accuracy of the created cases in relation to the real-world situations they represented. Each case included the patient’s name, age, COPD diagnosis, baseline values for oxygen saturation, heart rate, and breathing rate, as well as current values for these parameters. Any questionnaires that might be completed by the patient, such as the CERT and EXACT, were also included. Contextual details, such as relevant medical history, were incorporated to closely align the data with the intended user group: patients with stable COPD, permitted to self-monitor vital measurements at home until deterioration of their health prompts follow-up treatment by their care professionals. The six cases are summarized in Table 1 . Table 1 Titles and descriptions of the fully synthetic patient cases constructed in a knowledge-driven approach and in close collaboration with an experienced pulmonologist. Case nr. Case title Description 1 Measurement error Measurements taken by the patient indicate more severe symptoms than their answers to the questionnaire would indicate. Measurement devices could be misused or defective. 2 Classic acute exacerbation A patient has severe symptoms that prevent them from performing regular tasks around the house. 3 Exacerbation over multiple days A patient’s condition deteriorates over four days after an initial exacerbation. 4 Medication may be stopped A patient is showing improvement after multiple days of worsening conditions. The measurements have returned to their baseline values. 5 Mild chronic symptoms A patient indicates they are suffering from long-term, mild symptoms. 6 Severe chronic symptoms A patient indicates they are suffering from long-term, severe symptoms. Characteristics of participants Participants were recruited from the Department of Respiratory Medicine at the MUMC + hospital through convenience sampling. The other inclusion criteria were having at least one year of experience at a respiratory medicine department (either at the MUMC + or another hospital), being able to speak and read Dutch, being able to take part in the interview, and being willing and able to sign the informed consent form of their own volition. There were no separate criteria for exclusion. We did not include participants from other hospitals because the CKM was developed specifically based on the accepted clinical practices at the MUMC+. Potential participants were informed of the study through a presentation and three emails, and could afterwards indicate if they wished to participate. The presentation and emails consisted of information on the purpose and goals of the study, what was expected from participants, what data would be collected, and how it would be stored. Participants were not informed beforehand of the research questions nor the patient cases. 30 of the employees at the Department of Respiratory Medicine of the MUMC + were eligible for inclusion. Of these, 11 agreed to participate. 3 dropped out during the study due to unavailability, leaving 8 participants. Their characteristics are listed in Table 2 . Table 2 Summary of study population characteristics. IQR indicates the Interquartile Range. Calculated values are rounded to one decimal place. Characteristics No. of participants (total = 8) % of participants Professional groups Pulmonologists (incl. pulmonary oncologist) 4 50.0 Pulmonologists in training 2 25.0 Non-physicians with patient contact (physician assistant, nurse practitioner) 2 25.0 Age (median = 49.0, IQR = 25.0) 20–29 2 25.0 30–39 1 12.5 40–49 1 12.5 50–59 2 25.0 60 or more 2 25.0 Years of work experience (median = 11.0, IQR = 15.9) 0–14 4 50.0 15–29 2 25.0 30 or more 2 25.0 Interviews Eight individual in-depth interviews were performed by author EB in weeks 18 through 22 of 2024. Although an interview guide was used (which is available as supplementary material to this paper), topics of interest could be pursued during the conversation in between the prepared questions.( 13 ) This fit the variation we anticipated in responses to the patient cases and the CKM. We preferred individual interviews to a focus group to allow each participant to speak freely and without being influenced by colleagues, so we might elicit any interprofessional differences. Audio recordings of the interviews were made to aid analysis and avoid the need for field notes. The interviews were divided into two parts. First, participants were sequentially presented with cases 1 –3 from Table 1 . They were given time to study the patient’s data and were then asked to respond with the recommendations they would give this patient, as if it were a real patient. This open format was used to mimic a real-world scenario as closely as possible, which would not be possible if the participant needed to choose from prepared answers. Secondly, participants were presented with cases 4 –6 together with the recommendations previously generated by the CKM. Instead of being asked to provide recommendations themselves, participants were to indicate whether they agreed with the recommendations and why. Data analysis The authors EB and EE inductively coded all relevant themes from the interview data, following Thomas’ method.( 14 ) Discrepancies in codes and themes were discussed between the coders to reach consensus. The interview data on cases 1 –3 resulted in themes and responses concerning the clinical recommendations by the participants. To quantify any differences between the CKM-generated recommendations and that provided by our participants, we determined how often the major themes identified by the coding of the two sets of recommendations were not the same. The interview data on cases 4 –6 produced themes on the participants’ attitude towards the CKM. We also determined how often participants adopted the CKM’s recommendations or not and for what reasons. Results Cases 1-3: Themes regarding similarity of clinical recommendations Concerning the comparison between the recommendations given by participants and the CKM, the coding process produced six high-level themes and twenty-four related responses. These are displayed with frequencies in Table 3 . We will briefly discuss the main findings per theme. Table 3: Frequency of all themes and responses extracted from the recommendations provided by participants and the CKM for synthetic patient cases 1-3. Calculated values are rounded to one decimal place. CKM-generated recommendations are marked with *. themes and responses Case 1 case 2 case 3 Retake measurement 2 min 5 min 30 min 1 hour After inhalation medication 7 (87.5%) * 1 (12.5%) 2 (25.0%) 4 (50.0%) 1 (12.5%) 1 (12.5%) 1 (12.5%) 0 (0.0%) Additional questions Fever Exertion Environmental and external factors (e.g. outside/inside, daily routine, allergies, travelling abroad, contact with animals) Swelling or redness of leg Sputum appearance (e.g. color, consistency) Deviation from normal symptoms Type of chest pain Onset of symptoms (e.g. gradual, sudden) Urology symptoms (e.g. change in defecation, urinary tract infection) Weight Medical interventions (e.g. surgery, chemotherapy) Smoking habit Cardiac history Patient’s assessment of their situation 6 (75.0%) 3 (37.5%) 6 (75.0%) 1 (12.5%) 8 (100.0%) 5 (62.5%) 2 (25.0%) 2 (25.0%) 5 (62.5%) 4 (50.0%) 1 (12.5%) 3 (37.5%) 2 (25.0%) 2 (25.0%) 1 (12.5%) 1 (12.5%) 1 (12.5%) 6 (75.0%) 6 (75.0%) 2 (25.0%) 2 (25.0%) 4 (25.0%) 3 (37.5%) 2 (25.0%) 1 (12.5%) 1 (12.5%) 1 (12.5%) 1 (12.5%) 1 (12.5%) Contact with care professional Consultation by phone Check-up appointment Diagnostics (e.g. x-ray, MRI, lab tests) 2 (25.0%) 2 (25.0%) 8 (100.0%) 5 (62.5%) * 1 (12.5%) 4 (50.0%) * 8 (100.0%) 4 (50.0%) * 6 (75.0%) * Medication Increase inhaler dose Start new medication or antibiotic treatment 0 (0.0%) 3 (37.5%) 1 (12.5%) * 2 (25.0%) 3 (37.5%) * 3 (37.5) Continuation of monitoring 1 (12.5%) 0 (0.0%) * 0 (0.0%) * Lifestyle advice 0 (0.0%) 0 (0.0%) * 0 (0.0%) * Retake measurement This theme was most frequently mentioned in advice for case 1 , with seven out of eight participants (87.5%) recommending retaking the measurement. Participants advised measuring the vitals again to verify if the initial measurement was correct, possibly taken after a physically straining activity. The responses included various suggested time intervals between the first and second measurements, ranging from 5 minutes to 1 hour. One participant recommended retaking the measurement after using inhalation medication. The model advised measuring the vitals again after 2 minutes, which was the shortest interval of all recommendations. Additional questions Throughout all three cases in the first half of the interviews, many participants sought additional information about the patient. When referring to environmental or external factors, participants meant any outside influence on the patient's health. These factors included the location of the measurement (e.g. inside or outside), the patient’s daily routine, recent travel, contact with animals, allergy flare-ups, and exposure to sick individuals in their direct environment (e.g. at home). Deviations from normal symptoms were frequently inquired about because COPD patients often have certain consistent symptoms, such as shortness of breath during household activities. Understanding how current symptoms differ from usual ones can help gauge the severity of the situation. Finally, one participant asked for the patient’s self-assessment, inquiring if they could manage treatment at home while still taking care of themselves (e.g. eating and washing). Contact with care professional The recommendations by both the CKM and the participants revealed three responses related to contact with care professionals: phone consultations, (face-to-face) check-up appointments, and in-hospital diagnostics. In cases 2 and 3, all participants emphasized the importance of patient contact. For case 1 , two respondents (25%) suggested a phone consultation to ask additional questions and assess speech and breathing patterns. Medication None of the participants included medication in their advice for case 1 , while three participants (37.5%) included it in cases 2 and 3. The CKM recommended increasing the existing inhaler dose for both these cases. However, nearly all the participants who mentioned medication in their advice preferred starting a new medication or antibiotic treatment instead. Continuation of monitoring While included by the CKM for both cases 2 and 3, only one participant included the continuation of monitoring in their advice for case 1 . This involved asking the patient to re-enter their vital measurements and complete the questionnaires the next day to ensure the initial measurement was not erroneous and the patient was not unknowingly unwell. Lifestyle advice The model included lifestyle advice for both cases 2 and 3. However, none of the participants mentioned it. Lifestyle advice included recommendations such as using airway clearance techniques, practicing breathing and mindfulness exercises, and slowing down the pace of life. Cases 4-6: Themes regarding the level of agreement and attitude We found seven major themes and twenty-seven subthemes related to participants’ level of agreement with the CKM-generated recommendations for cases 4 –6 and their attitude towards the CKM. These are displayed in Fig. 3 . While some themes appeared in both parts of the interviews, the responses differed due to the nature of the questions we asked. Actionable advice and reasons for adopting advice were not directly comparable. Medication Participants considered administering medications, such as inhalers or antibiotics. In case 4 , most agreed to continue the current medication, with half of the participants adding that no additional medication was necessary. For case 5, those adopting the advice agreed on increasing the inhaler dose. Among non-adopters, opinions were divided: two participants supported increasing the dose, while the other two did not. The need for additional information and clarification on which inhaler to increase were the main reasons for non-adoption. Lastly, in case 6, most participants agreed on the necessity of increasing the inhaler dose, even though they did not adopt the advice. Monitoring This theme was the most controversial, with responses reflecting differing opinions on the frequency of monitoring after entering a measurement in the CKM. The model suggested that patients should measure their vitals and fill in questionnaires the following day. Two participants recommended fewer measurements, adding that measurements only needed to be taken when symptoms arose. For case 4 , disagreements with the model’s advice on monitoring frequency were not perceived to be significant enough to prevent adoption of the overall advice. Symptoms This theme focused on observations based on the vital measurements and questionnaire responses provided by the patient. Participants used these observations to justify their decisions. In case 4 , if vitals returned to normal and the patient felt well, no additional medication was deemed necessary. Conversely, worsening symptoms in case 5 might warrant an increased inhaler dose. Symptoms were less significant in case 6. Additional questions Similar to the first part of the interviews, participants sought additional information, though this theme was less prevalent in this second part. The responses were a subset of those identified during the first part. Despite its lower frequency, the theme significantly influenced responses. For example, in case 5, the need for additional information was important enough to reject the model’s advice. Contact with care professional This theme primarily addressed the role of the care professional in the treatment process. In case 6, the model recommended that patients initiate contact with their care professional and should undergo laboratory tests. However, participants disagreed with this approach, unanimously asserting that decisions regarding further treatment steps should be left to the care professional. While some participants agreed that laboratory tests should be performed, most suggested that it should not be included as recommendation to allow the treating professional to determine whether the tests are necessary or not. Lifestyle advice The responses varied similarly among participants regarding the lifestyle advice provided by the model in case 6, reflecting diverse perspectives. While a majority acknowledged the potential usefulness of breathing exercises, they generally perceived mindfulness exercises as less applicable. A few participants expressed concerns that patients experiencing anxiety from an exacerbation might find it challenging to engage effectively with mindfulness exercises. Qualities of CKM-generated advice The final major theme identified pertained to the qualities participants’ attributed to the CKM-generated recommendations. Three responses were identified: unclear advice, excessive information in the advice, and potential confusion for the patient. There was consensus among participants, particularly evident in case 6, that the advice contained an excessive amount of information, potentially leading to confusion among patients. As previously mentioned, participants emphasized that if patients are advised to contact their care professional, subsequent treatment decisions should be left to the professional’s discretion. This concern was a primary factor influencing participants’ decision not to adopt the advice in case 6. Knowledge elicitation from expert versus CKM recommendations The difference between the CKM-generated recommendations and those expressed by participants was quantified for cases 1 –3 by counting the number of deviations between the major themes. Deviations included instances where the participant either omitted advice recommended by the model or added advice not contained in the model’s recommendations. The total difference score is a combination of all participants’ difference scores to allow for case-by-case comparison. Case 2 had the highest total difference score (29) and mean difference score (3.6 SD 0.7), indicating the greatest divergence from the model’s recommendations. The number of differences varied per case and theme. Across all three cases, participants asked additional questions in almost every interview (20 of 24 cases, 83.3%). Continuation of monitoring was absent in the participants’ advice for cases 2 and 3, but was included once in case 1 . Similarly, lifestyle advice was not present in the participants’ recommendations for cases 2 and 3. Medication was included by three participants in cases 2 and 3, but never for case 1 . The themes with the fewest deviations from the model’s advice were retake measurement and contact with care professional , each with only two differences. Retaking the measurement was consistently included in case 1 when recommended by the model. Contact with the care professional was always advised in cases 2 and 3 when the model recommended it. A comprehensive overview of the differences per major theme, the absolute deviation score, and mean deviation score, is presented in Table 4 . Table 4 Differences per major theme, split by case and in total. For each case, the left column shows the frequency of each theme as mentioned by participants (P). The right column compares the frequency of these themes in the participants’ recommendations with the CKM’s recommendations. A positive number indicates the theme was present in the participants’ recommendations but not in the CKM’s, while a negative number indicates the theme was in the CKM’s recommendations but not in the participants’. The total deviation score is the absolute sum of these differences. The mean deviation score is the total absolute deviation score divided by the number of participants. Calculated values are rounded to one decimal place. CKM-generated recommendations are marked with *. Case 1 case 2 case 3 themes Ps +/- Ps +/- Ps +/- Total difference score Retake measurement 7 * -1 1 + 1 0 0 2 Additional questions 6 + 6 8 + 8 6 + 6 20 Contact with care professional 2 + 2 8 * 0 8 * 0 2 Medication 0 0 3 * -5 3 * -5 10 Continuation of monitoring 1 + 1 0 -8 0 * -8 17 Lifestyle advice 0 0 0 -8 0 * -8 16 Absolute difference 10 30 27 Mean difference, standard deviation 1.3 SD 0.8 3.6 SD 0.7 3.4 SD 0.7 Interprofessional differences The second aim of this study was to examine potential interprofessional differences in treatment strategies among the participating care professionals. This was also done using difference scores. Figure 4 illustrates the absolute number of differences between the recommendations of the participants in the form of a heatmap, with darker colors indicating higher numbers of differences between the responses in the recommendations. The average number of differences per case was 3.4 (SD 1.3) for case 1 , 7.3 (SD 2.2) for case 2, and 5.6 (SD 2.2) for case 3, all rounded to one decimal place. The average number of interprofessional differences between and within professional groups is presented in Table 5 . For pulmonologists, the number of interprofessional clinical differences within the group was lower than the differences between pulmonologists and the other groups. Conversely, for pulmonologists in training and non-physicians, the differences within the group were higher than between groups. Notably, among non-physicians, the number of within-group differences was the highest. Table 5 Average number of differences between the three groups of care professionals (pulmonologists, pulmonologists in training, and non-physicians) across cases 1 –3. The diagonal presents the average number of differences between the care professionals within the same group. Calculated values are rounded to one decimal place. Pulmonologists Pulmonologists in training Non-physician with patient contact Pulmonologists 12.7 16.8 16.3 Pulmonologists in training (resident) 16.8 18.0 16.3 Non-physician with patient contact 16.3 16.3 19.0 Adoption of CKM-generated recommendations The adoption of the model’s advice varied across cases 4 –6. As shown in Fig. 3 , all care professionals adopted the advice for case 4 , despite disagreements about the recommended frequency of monitoring. For case 5, adoption was split evenly, with half of the care professionals agreeing with the model’s advice and the other half disagreeing, primarily due to differing opinions on medication administration. In case 6, nearly all participants rejected the model’s advice, except for one. They criticized the advice for containing too much information, which could confuse the patient or complicate the care professional’s decision-making process. One professional noted, “ If you say ‘contact your care professional’, the rest of the treatment plan should be left up to that care professional. ” Recommending laboratory tests could limit the care professional’s ability to consider alternative actions, as patients might assume lab tests are essential for proper treatment. Participants with the same professional role tended to accept the same cases. While all the pulmonologists accepted the model’s advice for case 5, both of the other groups of professionals (pulmonologists in training and non-physicians with patient contact) did not accept the advice. The acceptance behavior for the pulmonologists in training and the non-physicians was identical, both within and between the groups. The pulmonologists had clinical differences both within their group, as well as between them and the other two groups. Figure 5 displays how acceptance of CKM-generated recommendations increased with the age of participants. Discussion The concept of decision support for the (self-)management of COPD exacerbations is well-established, with several alternatives to the COPD CKM having been developed.( 15 , 16 ) The validation of these tools, however, varies significantly. The COPDPredict™ application, for example, was validated by demonstrating its usability for both patients and physicians, and by showing that its prediction algorithm can reliably inform both user groups about acute events.( 16 ) This was achieved by comparing “algorithm-defined” exacerbations to “clinician-defined” exacerbations. As this validation method was established to properly validate a COPD monitoring tool, our study applied a similar approach. It compared the automated output of the COPD clinical knowledge model to human-generated output to ensure the model aligns with good clinical practice. This approach helps prevent a decline in care quality as healthcare becomes more automated. In contrast, the ACCESS prediction model was validated using statistical methods, such as an ROC-analysis.( 15 ) Although the model was developed with input from pulmonologists, its evaluation did not include validation with other pulmonary specialists. A later evaluation of the ACCESS software focused on the validity of its most important treatment advice (contact the healthcare professional).( 17 ) Our current study evaluated the entire advice rather than just a subset. The results provide insight into possible areas of domain knowledge that require further elicitation to expand the usability of the CKM. They also highlight interprofessional clinical differences, and demonstrate the rate of adoption among care professionals based on a variety of characteristics. To the authors’ knowledge, this is the first study to evaluate the human versus CKM performance in COPD monitoring based on synthetic patient data. A principal finding of this study was the significant variation between the model’s and care professionals’ recommendations across all three cases. Overall, the participants omitted more than half of the responses included in the model’s advice. At the same time, they consistently sought more patient information, particularly regarding body temperature, exertion, and sputum appearance. One possible explanation is that the participants felt they needed more information to thoroughly understand their patient’s status. Another possibility is that care professionals are accustomed to different ways of collecting information and taking decisions. If so, the CKM’s current presentation might not fit these habits. Further user-centered research may shed light on whether adjustments to the CKM and/or the information collecting and processing habits of care professionals might be beneficial for their collaboration. Participants also tended to prescribe more impactful treatments than the model recommended. While the model suggested increasing the inhaler dose, some participants preferred starting antibiotic treatments, indicating that the model’s recommendations were perceived to be insufficient for their patient’s needs. Both the need for additional information and the treatment manner suggest that the care professionals took a more cautious approach than the model when treating a patient. While a cautious approach is not necessarily problematic, it could lead to overtreatment. In recent work, it has been found that overtreatment of COPD occurs in almost one in two cases, meaning that this is a prevalent issue within COPD care.( 18 ) The CKM could prevent overtreatment by giving physicians the confidence to work with less information and to only seek additional information when it is required for decision making. At the same time, it may also prompt the use of information that is currently often neglected, such as spirometry in the diagnostic phase of COPD. This is a piece of information with high specificity that can prevent the erroneous diagnose of COPD, but is not always considered.( 18 , 19 ) Overall, the rich results of comparing the expert recommendations with the CKM’s indicate that case-based interviewing is a suitable technique for knowledge elicitation regarding clinical knowledge. This is important, because CKMs and their performance are entirely based on available domain knowledge. In this, they are distinct from data-driven solutions, such as machine learning, in which new knowledge is encoded in the model by having it “learn” from large quantities of data.( 20 ) Such data-driven knowledge has its place in medicine, for example in predictive models and image processing, but can be unsatisfying for healthcare professionals who require more explainability of model-generated recommendations, either for themselves or for their patients.( 21 ) CKMs allow tracing each recommendation back to all the rules and inputs that led to the recommendation being generated. A disadvantage of this approach for CKMs is that they are unable to generate new rules based on available data. A possible architecture to mitigate this would be a two-agent system, in which the CKM is complemented by a second data-driven component. This second component might generate new rules through induction, by observing the recommendations from the CKM, the registered actions taken by healthcare professionals, and patient outcomes, forming a closed-loop system. The elicited knowledge revealed evident interprofessional clinical differences between different types of care professionals and within groups of the same type. This forms a significant hurdle in expanding the accepted knowledge encoded in the CKM. To the authors’ knowledge, no prior studies have specifically examined differences among pulmonary care professionals treating COPD patients. A recent publication by Svensson and Jacobsson highlighted inconsistencies in decision-making among emergency department care professionals.( 22 ) Their framework identified three modes of medical decision-making: experiential-based (emotional and intuitive), ostensive-based (rational and rule-based), and action-based (physician’s doings in the situation). The interplay between these decision-making modes varies between care professionals, leading to an eight-fold typology of medical decision-making praxis. This suggests that interprofessional differences in medical decision-making are not unique to the pulmonary specialists included in this study and warrants further research to identify the causes of these discrepancies. Additionally, our method resulted in material that the participants and their colleagues can use to further discuss their professional differences. This is valuable, even outside of their possible continued use of the CKM. The third objective of this research was to examine the adoption behavior of care professionals and whether characteristics such as job experience and age influence this behavior. Overall, the proposed advice from the model was adopted by participants about half of the time, indicating a need for further consultation with more professionals during model development to achieve greater consensus. Both characteristics – job experience and age – impacted the adoption of the model’s advice. The trend that the experienced professionals accepted the model’s advice more readily than other groups contrast with prior research. Adler-Milstein and colleagues have suggested that experienced professionals might distrust AI tools, especially if the recommendations contradict their intuitive conclusions.( 23 ) Similarly, studies on automation bias indicate that less experienced professionals are more prone to over-relying on automation, suggesting that experienced professionals would be less likely to adopt AI-generated recommendations.( 24 , 25 ) The contrast between this study’s findings and existing literature highlights the need for further research to understand the correlation between a care professional’s experience and the adoption of automatically generated recommendations. Given that the more experienced participants were older, it is challenging to determine whether experience or age has a greater effect on adoption rates. Nonetheless, the results suggest that adoption rates increase with age. This contradicts several studies suggesting that younger individuals are more likely to adopt AI tools due to their generally techno-positive attitude, which aligns with the idea that the impact of age decreases as experience increases.( 26 – 28 ) A larger and more varied study population would be required to conclusively determine whether age is correlated with the adoption rate of automatically generated recommendations. Finally, some adoption behaviors showed consensus among professionals. For instance, nearly all care professionals rejected the advice for case 6 due to excessive information. Conversely, a theme that caused discussion but did not influence the adoption rate was the frequency of monitoring. This suggests that as long as the recommendations appear sensible, care professionals accept a certain degree of interprofessional differences in the specific execution of the advice. A significant strength of this study is its inclusion of the intended user group in the evaluation process. By involving care professionals who will use the CKM, the study facilitates their gradual familiarization with the tool before its implementation. Additionally, involving care professionals in the tool’s development fosters a sense of responsibility for its outcome and implementation, potentially increasing its adoption when it is eventually ready to be integrated into COPD care. Patients were intentionally excluded from this preliminary evaluation of the CKM to first determine if the model aligned with current medical practice. Future studies should include both patients and care professionals to assess the tool’s usability. Conclusions Case-based interviews proved an effective method for eliciting clinical knowledge required for completing a CKM. Themes that emerged included retaking measurements, asking the patient additional questions, contacting the care professional, medication, continuation of monitoring, and lifestyle advice. In many of their responses, participants exhibited more caution than the CKM, seeking additional information and proposing stronger measures, such as medication. Interprofessional differences were found between different types of care professionals, as well as within groups of the same type. Pulmonologists were most unanimous in their advice, while variations between all three groups were similar. The care professionals showed a moderate willingness to adopt the CKM, with the proposed recommendations accepted slightly more than half the time. This suggests that further collaboration with care professionals is necessary to develop advice that gains wider acceptance. Higher adoption rates were associated with older, more experienced professionals, although the interplay between these characteristics and their precise influence on adoption rates remains inconclusive. Abbreviations CERT COPD Exacerbation Recognition Tool CKM clinical knowledge model COPD chronic obstructive pulmonary disease EXACT Exacerbations of Chronic Pulmonary Disease Tool MUMC+ Maastricht University Medical Center+ SD standard deviation Declarations Ethics approval and consent to participate This study was conducted in accordance with the ethical principles of the Netherlands Code of Conduct for Research Integrity (2018). All participants signed an informed consent form agreeing with participating in the research described in this paper. All data collected was pseudonymized, with only the author who conducted the interviews having the key. All recordings of interviews were destroyed after analysis. The Research Ethics Committee of the Faculty of Health, Medicine and Life Sciences of Maastricht University approved the research described in this paper under case number FHML/HDT/2024.002. Consent for publication All participants signed an informed consent form agreeing with the publication of their data in this paper. Availability of data and materials The interview data are available from the corresponding author on reasonable request. Competing interests FT, PRM, WB, and HtB are shareholders of Open Walnoot, a company that specializes in providing the techniques and solutions described in this paper to various clients within the healthcare sector. Funding No supplemental funding was received for the execution of this research project. Author’s contributions EB and EE executed the majority of the research and drafted the manuscript, with significant support from SS, MS, and FvM. FT, PRM, WBa, and HtB constructed the clinical knowledge model in collaboration with SS. FT, PRM, WBa, and HtB all contributed equally in sharing their relevant materials, experience and insights. All authors provided additions and modifications to the draft versions of the manuscript. Acknowledgements Not applicable. References Edelman E, Tijssen F, Munniksma PR, Bast W, Bohmer H, van ten, Eldik N et al Clinical knowledge modeling: An essential step in the digital transformation of healthcare. The Innovation [Internet]. 2024 Nov 4 [cited 2025 Feb 28];5(6). Available from: https://www.cell.com/the-innovation/abstract/S2666-6758(24)00156-5 Klemann D, Winasti W, Tournois F, Mertens H, van Merode F (2024) Quantifying the Resilience of a Healthcare System: Entropy and Network Science Perspectives. Entropy 26(1):21 Galbraith JR (1974) Organization Design: An Information Processing View. Interfaces 4(3):28–36 Patel VL, Arocha JF, Kaufman DR (1999) Expertise and Tacit Knowledge in Medicine. Tacit Knowledge in Professional Practice. Psychology Wickramasinghe N, Davison G (2004) Making Explicit the Implicit Knowledge Assets in Healthcare: The Case of Multidisciplinary Teams in Care and Cure Environments. Health Care Manag Sci 7(3):185–195 Ting SL, Wang WM, Tse YK, Ip WH (2011) Knowledge elicitation approach in enhancing tacit knowledge sharing. Ind Manag Data Syst 111(7):1039–1064 de Gans ST, Maessen GC, van de Pol MHJ, van Apeldoorn MJ, van Ingen-Stokbroekx MAL, van der Sloot N et al (2023) Effect of interprofessional and intraprofessional clinical collaboration on patient related outcomes in multimorbid older patients – a retrospective cohort study on the Intensive Collaboration Ward. BMC Geriatr 23(1):1–11 Jones PW, Wang,Chanzheng C et al (2022),Ping, Chen, Liping, Wang, Daoxin, Xia, Junbo,. The Development of a COPD Exacerbation Recognition Tool (CERT) to Help Patients Recognize When to Seek Medical Advice. Int J Chron Obstruct Pulmon Dis. ;17:213–22 Leidy NK, Wilcox TK, Jones PW, Murray L, Winnette R, Howard K et al (2010) Development of the EXAcerbations of Chronic Obstructive Pulmonary Disease Tool (EXACT): A Patient-Reported Outcome (PRO) Measure. Value Health 13(8):965–975 Raghunathan TE (2021) Synthetic Data. Annu Rev Stat Its Appl. ;8(Volume 8, 2021):129–40 McLachlan S Realism in synthetic data generation Murtaza H, Ahmed M, Khan NF, Murtaza G, Zafar S, Bano A (2023) Synthetic data generation: State of the art in health care domain. Comput Sci Rev 48:100546 Qu SQ, Dumay J (2011) The qualitative research interview. Qual Res Acc Manag 8(3):238–264 Thomas DR (2006) A General Inductive Approach for Analyzing Qualitative Evaluation Data. Am J Eval 27(2):237–246 van der Heijden M, Lucas PJF, Lijnse B, Heijdra YF, Schermer TRJ (2013) An autonomous mobile system for the management of COPD. J Biomed Inf 46(3):458–469 Patel N, Kinmond,Kathryn, Jones, Pauline, Birks, Pamela, and, Spiteri MA (2021) Validation of COPDPredict ™ : Unique Combination of Remote Monitoring and Exacerbation Prediction to Support Preventative Management of COPD Exacerbations. Int J Chron Obstruct Pulmon Dis. ;16:1887–99 Boer LM, van der Heijden ME, Lucas, Peter JF, Vercoulen, Jan H et al (2018) Assendelft, Willem JJ,. Validation of ACCESS: an automated tool to support self-management of COPD exacerbations. Int J Chron Obstruct Pulmon Dis. ;13:3255–67 Fiore M, Ricci M, Rosso A, Flacco ME, Manzoli L (2023) Chronic Obstructive Pulmonary Disease Overdiagnosis and Overtreatment: A Meta-Analysis. J Clin Med 12(22):6978 Heffler E, Crimi C, Mancuso S, Campisi R, Puggioni F, Brussino L et al (2018) Misdiagnosis of asthma and COPD and underuse of spirometry in primary care unselected patients. Respir Med 142:48–52 Dubois D, Hájek P, Prade H (2000) Knowledge-Driven versus Data-Driven Logics. J Log Lang Inf 9(1):65–89 Reddy S (2022) Explainability and artificial intelligence in medicine. Lancet Digit Health 4(4):e214–e215 Svensson M, Jacobsson M (2024) Managing inconsistencies in medical decision-making: An eight-fold typology. Eur Manag J 42(1):130–141 Adler-Milstein J, Aggarwal N, Ahmed M, Castner J, Evans BJ, Gonzalez AA et al (2022) Meeting the Moment: Addressing Barriers and Facilitating Clinical Adoption of Artificial Intelligence in Medical Diagnosis. NAM Perspect. 10.31478/202209c Dratsch T, Chen X, Rezazade Mehrizi M, Kloeckner R, Mähringer-Kunz A, Püsken M et al (2023) Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance. Radiology 307(4):e222176 Goddard K, Roudsari A, Wyatt JC (2014) Automation bias: empirical results assessing influencing factors. Int J Med Inf 83(5):368–375 Huisman M, Ranschaert E, Parker W, Mastrodicasa D, Koci M, Pinto de Santos D et al (2021) An international survey on AI in radiology in 1,041 radiologists and radiology residents part 1: fear of replacement, knowledge, and attitude. Eur Radiol 31(9):7058–7066 Khanijahani A, Iezadi S, Dudley S, Goettler M, Kroetsch P, Wise J (2022) Organizational, professional, and patient characteristics associated with artificial intelligence adoption in healthcare: A systematic review. Health Policy Technol 11(1):100602 Rushlow DR, Croghan IT, Inselman JW, Thacher TD, Friedman PA, Yao X et al (2022) Clinician Adoption of an Artificial Intelligence Algorithm to Detect Left Ventricular Systolic Dysfunction in Primary Care. Mayo Clin Proc. ;97(11):2076–85 Additional Declarations Competing interest reported. FT, PRM, WB, and HtB are shareholders of Open Walnoot, a company that specializes in providing the techniques and solutions described in this paper to various clients within the healthcare sector. Supplementary Files InterviewGuide.docx Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 28 Mar, 2026 Reviews received at journal 27 Mar, 2026 Reviewers agreed at journal 06 Mar, 2026 Reviewers agreed at journal 01 Feb, 2026 Reviews received at journal 06 Jul, 2025 Reviewers agreed at journal 12 Jun, 2025 Reviewers invited by journal 11 Jun, 2025 Editor assigned by journal 11 Jun, 2025 Editor invited by journal 09 Jun, 2025 Submission checks completed at journal 06 Jun, 2025 First submitted to journal 06 Jun, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6819489","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":471251875,"identity":"d7c5bc7b-be44-4693-bac8-1a77711839e5","order_by":0,"name":"Eric Edelman","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABRElEQVRIie3RTUsCQRjA8RFh9vKsex1Z0q/wiFBJhV+lwataEISnWBDGi+U1ib7DgiB1W/HgZeyszEXxWMF2CYM55LpRtm1Bt4j9Hx6Yl99chpCkpD8aI8eraTjhihIIN4O8YGTiCK4mrI8xICnnnXjrR74h7DAkJCRvxRCLpAcTH/v5new995daH2WM8Whh3uzyjj2aD58EqUdI1qGV0iWqwu1VrZc9F1gSUOdNUzLevaiiNxDkJELQg20bUKVcVXOZ6SBSBoWmKVgRJZCAcPGZlD3r2daoyu503HvReoOUpTGLI0iA2gQVdydm3wb6QXIIBOMIG9JiqY2q4spaf29LFJFCtdC9XhEmAT15x6LEajXnk2VDHbijcW/6oHOYb0n0H8UZWG1j4TdO93kn8inpr/+0WYqyny/EpH8tkpKSkv5dr5m8c2TXTS95AAAAAElFTkSuQmCC","orcid":"","institution":"Maastricht University","correspondingAuthor":true,"prefix":"","firstName":"Eric","middleName":"","lastName":"Edelman","suffix":""},{"id":471251881,"identity":"b97e60e9-2c83-42fe-9014-61d393c872a0","order_by":1,"name":"Esmee Bellemakers","email":"","orcid":"","institution":"Maastricht University","correspondingAuthor":false,"prefix":"","firstName":"Esmee","middleName":"","lastName":"Bellemakers","suffix":""},{"id":471251885,"identity":"2c3d1cca-65db-4f3c-b2a2-132a2a088e4c","order_by":2,"name":"Fabian Tijssen","email":"","orcid":"","institution":"Maastricht University Medical Center+","correspondingAuthor":false,"prefix":"","firstName":"Fabian","middleName":"","lastName":"Tijssen","suffix":""},{"id":471251888,"identity":"d2c7ede8-a7fa-484d-931b-e44f3733f3ca","order_by":3,"name":"Popke Rein Munniksma","email":"","orcid":"","institution":"Open Walnoot","correspondingAuthor":false,"prefix":"","firstName":"Popke","middleName":"Rein","lastName":"Munniksma","suffix":""},{"id":471251890,"identity":"5b3b5863-91b6-497f-8b63-f0d2571865ec","order_by":4,"name":"Wim Bast","email":"","orcid":"","institution":"Open Walnoot","correspondingAuthor":false,"prefix":"","firstName":"Wim","middleName":"","lastName":"Bast","suffix":""},{"id":471251892,"identity":"94d6b497-6755-4e98-b0e7-7baf2ddd1a68","order_by":5,"name":"Harold ten Bohmer","email":"","orcid":"","institution":"Open Walnoot","correspondingAuthor":false,"prefix":"","firstName":"Harold","middleName":"ten","lastName":"Bohmer","suffix":""},{"id":471251895,"identity":"58a65981-3f9e-4588-b4f3-e4c9666c0f5f","order_by":6,"name":"Sami O. Simons","email":"","orcid":"","institution":"Maastricht University","correspondingAuthor":false,"prefix":"","firstName":"Sami","middleName":"O.","lastName":"Simons","suffix":""},{"id":471251896,"identity":"5128329a-1cca-4d51-ad0a-053d5bcc5488","order_by":7,"name":"Marieke Spreeuwenberg","email":"","orcid":"","institution":"Maastricht University","correspondingAuthor":false,"prefix":"","firstName":"Marieke","middleName":"","lastName":"Spreeuwenberg","suffix":""},{"id":471251897,"identity":"be91c09e-e0b5-4b30-8108-696027920876","order_by":8,"name":"Frits van Merode","email":"","orcid":"","institution":"Maastricht University","correspondingAuthor":false,"prefix":"","firstName":"Frits","middleName":"van","lastName":"Merode","suffix":""}],"badges":[],"createdAt":"2025-06-04 10:38:29","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6819489/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6819489/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":84838428,"identity":"14652132-1242-4c5b-bd21-0ff1ccebd913","added_by":"auto","created_at":"2025-06-18 00:21:50","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":17873,"visible":true,"origin":"","legend":"\u003cp\u003eThe aim of the research described in this paper was to study whether early modeling of medical knowledge in a CKM and the fictional application of the resulting model together with clinical experts might enable us to elicit previously hidden domain knowledge. In a series of interviews based on synthetic patient cases, we gathered data from participating clinical experts. For cases 1-3, this concerned their clinical recommendations for the same cases. This allowed us to make a comparison with the CKM-generated recommendations, providing insights into 1. the quantity and quality of additional domain knowledge this method provided and 2. the level of interprofessional variation in clinical recommendations. Additionally, we calculated the adoption rate by the experts of the CKM-generated recommendations for cases 4-6, and coded the recurring themes in the experts’ responses to these recommendations. This resulted in insights into the experts’ attitude towards the CKM and its recommendations.\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-6819489/v1/ab660e7a82a9a3d391ad71bc.png"},{"id":84838429,"identity":"b6adbf3b-e7be-4db7-ad0e-b1c3e65e3b7d","added_by":"auto","created_at":"2025-06-18 00:21:50","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":21841,"visible":true,"origin":"","legend":"\u003cp\u003eVarious daily and optional inputs are fed to the clinical knowledge model. The model's decision rules process this data and produce the appropriate recommendations for the patient. These might include triggering the optional questionnaires.\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-6819489/v1/b70f7954b5a8e108fe97ab7a.png"},{"id":84838933,"identity":"0c0c7dcd-5291-49b6-bee1-072cd4e77a93","added_by":"auto","created_at":"2025-06-18 00:37:50","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":277310,"visible":true,"origin":"","legend":"\u003cp\u003eAll themes and subthemes extracted from the responses provided by participants to the CKM-generated recommendations for synthetic patient cases 4-6. CKM-generated recommendations are described at the top. The overall acceptance rate of each case’s recommendations follows below. Finally, a breakdown of each theme is displayed per group. Each subtheme is marked green if it conveyed a positive sentiment (agreement or justification of agreement) or red if it was negative (disagreement or justification of disagreement).\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-6819489/v1/b3d4e728787546d3954ffe76.png"},{"id":84838432,"identity":"8ac4cc18-894d-4346-b5aa-e40badb71fca","added_by":"auto","created_at":"2025-06-18 00:21:50","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":218341,"visible":true,"origin":"","legend":"\u003cp\u003eHeatmaps illustrating the absolute number of differences in subthemes among the recommendations by participants. Darker colors indicate more difference between the participants on the x and y axes.\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-6819489/v1/c79735a40faea29e74e71fce.png"},{"id":84838934,"identity":"fa5b995a-9e82-425a-be61-a6db654d3fad","added_by":"auto","created_at":"2025-06-18 00:37:50","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":22575,"visible":true,"origin":"","legend":"\u003cp\u003ePercentage of accepted CKM-generated recommendations versus participant age in years.\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-6819489/v1/41f5046cb9b63cef430083b1.png"},{"id":84839351,"identity":"18bbe2c8-ba08-4d4b-b419-f4872936b1e2","added_by":"auto","created_at":"2025-06-18 00:45:51","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1666785,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6819489/v1/559d9343-9a82-48d4-9da3-77c6d3b90b87.pdf"},{"id":84838769,"identity":"c6cbe5a8-00ce-47d6-bdee-755c56d611e9","added_by":"auto","created_at":"2025-06-18 00:29:50","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":798556,"visible":true,"origin":"","legend":"","description":"","filename":"InterviewGuide.docx","url":"https://assets-eu.researchsquare.com/files/rs-6819489/v1/c8761e8c31da992542e612e5.docx"}],"financialInterests":"Competing interest reported. FT, PRM, WB, and HtB are shareholders of Open Walnoot, a company that specializes in providing the techniques and solutions described in this paper to various clients within the healthcare sector.","formattedTitle":"Capturing Clinical Knowledge: The Digital Modeling of Expert Knowledge Concerning the Care for Patients with Chronic Obstructive Pulmonary Disease","fulltext":[{"header":"Background","content":"\u003cp\u003eAutomation and decision support in healthcare may be enabled by encoding medical knowledge in digital \u003cem\u003eclinical knowledge models\u003c/em\u003e (CKMs).(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e) Clinical knowledge modeling consists of two parallel activities: one is determining which decisions are made in the domain being modeled and which information is required to make them, and the other constitutes the conceptual modeling of the domain knowledge (i.e. how the information leads to the ability to make decisions). Examples of such decisions are whether a patient requires surgery, whether they are eligible for discharge, whether they are at risk of osteoporosis, which medication to prescribe, what triage category they should be assigned, etc.\u003c/p\u003e \u003cp\u003eWhile making single decisions in isolation is straightforward, healthcare often requires a multitude of interrelated decisions to be made at the same time, creating a complex network of required information and rules on its processing. Klemann et al. have described the crucial role of information processing in creating a robust and resilient healthcare system.(\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e) Following their reference to Galbraith’s information processing view on organizational design, a CKM greatly increases the information processing capabilities of a healthcare system. It does this by making the network of rule-based clinical decisions manageable and applicable. Once provided with input, all modeled decisions are continuously executed by the CKM. The final output is generated based on the collected outcomes of those decisions. The model’s output can therefore consist of multiple aspects, such as a request for the patient to fill in another questionnaire, as well as treatment recommendations for the care professional, such as the suggestion to increase a medication dose.\u003c/p\u003e \u003cp\u003e Finding the decisions and required information relevant to a CKM generally starts by analyzing existing protocols, guidelines, and any other formalized sources of knowledge on the topic at hand. However, this is usually not sufficient to construct a complete model. Clinical experts may also hold relevant, implicit knowledge that they have gained through experience or from other unrecorded sources.(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan additionalcitationids=\"CR5\" citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e–\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e) The aim of the research described in this paper was to study whether early modeling of medical knowledge in a CKM and the fictional application of the resulting model together with clinical experts might enable us to elicit this semi-hidden, but highly relevant domain knowledge.\u003c/p\u003e \u003cp\u003eSecondly, we looked for potential interprofessional differences in treatment strategies. Basing our definition on De Gans et al., interprofessional differences are differences among healthcare professionals from different professions, but within the same clinical specialty.(\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e) Investigating the presence of interprofessional clinical differences yields insights into the consistency and quality of the interpretation and execution of current clinical protocols. Any disagreements may require revisions of protocols or agreements amongst clinicians to restore consistency, which is essential for producing a CKM that is trusted by all.\u003c/p\u003e \u003cp\u003eFinally, we collected feedback from the professionals we included on further cases to study their attitude towards the CKM and its generated recommendations.\u003c/p\u003e \n\n "},{"header":"Methods","content":"\u003cp\u003eWe examined an example of clinical knowledge modeling that took place at the Department of Respiratory Medicine of the Maastricht University Medical Center+ (MUMC+). The resultant CKM is to support patients suffering from chronic obstructive pulmonary disease (COPD) in self-managing their chronic condition, thereby relieving pulmonary specialists from some of the monitoring and treatment of these patients. In a series of interviews based on synthetic patient cases, we gathered data from participating clinical experts. For cases \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e\u0026ndash;3, this concerned their clinical recommendations for the same cases. This allowed us to make a comparison with the CKM-generated recommendations, providing insights into 1. the quantity and quality of additional domain knowledge this method provided and 2. the level of interprofessional variation in clinical recommendations. Additionally, we calculated the adoption rate by the experts of the CKM-generated recommendations for cases \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e\u0026ndash;6, and coded the recurring themes in the experts\u0026rsquo; responses to these recommendations. This resulted in insights into the experts\u0026rsquo; attitude towards the CKM and its recommendations.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFigure 1 displays an overview of the study performed. Further details on the three major components follow below.\u003c/p\u003e\n\u003ch3\u003eCKM\u003c/h3\u003e\n\u003cp\u003eThe CKM was created in collaboration with the Department of Respiratory Medicine of the MUMC+. The experts that contributed to the model were not involved in the current study. Daily inputs were three vital parameters (blood oxygen saturation, heart rate, and respiratory rate) and the COPD Exacerbation Recognition Tool (CERT), which aims to increase a patient\u0026rsquo;s ability to report on their exacerbations.(\u003cspan class=\"CitationRef\"\u003e8\u003c/span\u003e) Patients could be requested to complete two additional questionnaires when needed: the Exacerbations of Chronic Pulmonary Disease Tool (EXACT) and a custom questionnaire.\u003c/p\u003e\n\u003cp\u003eEXACT is a patient-reported outcome measure for evaluating the frequency, severity, and duration of COPD exacerbations.(\u003cspan class=\"CitationRef\"\u003e9\u003c/span\u003e) It expands on topics also covered in the CERT questionnaire, such as increased intensity of coughing, increased amount of phlegm, shortness of breath, difficulty breathing, and impairment of performing activity. The model triggered EXACT once a patient\u0026rsquo;s symptoms exceeded a certain level of severity detected by the CERT.\u003c/p\u003e\n\u003cp\u003eThe second optional questionnaire was drafted by the experts who contributed to the model. This acute problem questionnaire posed questions about pain when breathing, body temperature, coughing up blood, palpitations, shortness of breath when lying down, and swollen ankles. These inputs were added to enable compliance with the Dutch national guidelines on the treatment of COPD.\u003c/p\u003e\n\u003cp\u003eAll these inputs jointly fed the decision rules included in the CKM, which were continuously executed by the application. After each evaluation, the model generated output based on the results. Figure\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e summarizes the functionality of the model.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eSynthetic patient cases\u003c/p\u003e\n\u003cp\u003eWe generated synthetic data for six patient cases that would be evaluated by both the CKM and our included healthcare professionals. Synthetic data has been described as fake data that faithfully represents the real-world data that is taken as its reference, making it fictional but realistic.(\u003cspan class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e11\u003c/span\u003e) We followed the method described by Murtaza and colleagues.(\u003cspan class=\"CitationRef\"\u003e12\u003c/span\u003e) They distinguish between data-driven, knowledge-driven, and hybrid approaches. In our case, we employed their knowledge-driven approach for the possibility to tailor the patient cases to specific scenarios without needing to look for real patients that matched the criteria. Collaboration with an experienced pulmonologist (who was not further involved with the study) across three in-person sessions ensured the accuracy of the created cases in relation to the real-world situations they represented.\u003c/p\u003e\n\u003cp\u003eEach case included the patient\u0026rsquo;s name, age, COPD diagnosis, baseline values for oxygen saturation, heart rate, and breathing rate, as well as current values for these parameters. Any questionnaires that might be completed by the patient, such as the CERT and EXACT, were also included. Contextual details, such as relevant medical history, were incorporated to closely align the data with the intended user group: patients with stable COPD, permitted to self-monitor vital measurements at home until deterioration of their health prompts follow-up treatment by their care professionals.\u003c/p\u003e\n\u003cp\u003eThe six cases are summarized in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n \u003cdiv class=\"colspec\" align=\"left\"\u003e\u0026nbsp;\u003c/div\u003e\n \u003ctable id=\"Tab1\" border=\"1\"\u003e\n \u003ccaption\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eTitles and descriptions of the fully synthetic patient cases constructed in a knowledge-driven approach and in close collaboration with an experienced pulmonologist.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCase nr.\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCase title\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eDescription\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMeasurement error\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMeasurements taken by the patient indicate more severe symptoms than their answers to the questionnaire would indicate. Measurement devices could be misused or defective.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eClassic acute exacerbation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eA patient has severe symptoms that prevent them from performing regular tasks around the house.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eExacerbation over multiple days\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eA patient\u0026rsquo;s condition deteriorates over four days after an initial exacerbation.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMedication may be stopped\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eA patient is showing improvement after multiple days of worsening conditions. The measurements have returned to their baseline values.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMild chronic symptoms\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eA patient indicates they are suffering from long-term, mild symptoms.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSevere chronic symptoms\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eA patient indicates they are suffering from long-term, severe symptoms.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eCharacteristics of participants\u003c/p\u003e\n\u003cp\u003eParticipants were recruited from the Department of Respiratory Medicine at the MUMC\u0026thinsp;+\u0026thinsp;hospital through convenience sampling. The other inclusion criteria were having at least one year of experience at a respiratory medicine department (either at the MUMC\u0026thinsp;+\u0026thinsp;or another hospital), being able to speak and read Dutch, being able to take part in the interview, and being willing and able to sign the informed consent form of their own volition. There were no separate criteria for exclusion. We did not include participants from other hospitals because the CKM was developed specifically based on the accepted clinical practices at the MUMC+.\u003c/p\u003e\n\u003cp\u003ePotential participants were informed of the study through a presentation and three emails, and could afterwards indicate if they wished to participate. The presentation and emails consisted of information on the purpose and goals of the study, what was expected from participants, what data would be collected, and how it would be stored. Participants were not informed beforehand of the research questions nor the patient cases.\u003c/p\u003e\n\u003cp\u003e30 of the employees at the Department of Respiratory Medicine of the MUMC\u0026thinsp;+\u0026thinsp;were eligible for inclusion. Of these, 11 agreed to participate. 3 dropped out during the study due to unavailability, leaving 8 participants. Their characteristics are listed in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\n \u003ctable id=\"Tab2\" border=\"1\"\u003e\n \u003ccaption\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eSummary of study population characteristics. IQR indicates the Interquartile Range. Calculated values are rounded to one decimal place.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCharacteristics\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eNo. of participants\u003c/p\u003e\n \u003cp\u003e(total\u0026thinsp;=\u0026thinsp;8)\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e% of participants\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eProfessional groups\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePulmonologists\u003c/p\u003e\n \u003cp\u003e(incl. pulmonary oncologist)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e50.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePulmonologists in training\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e25.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNon-physicians with patient contact\u003c/p\u003e\n \u003cp\u003e(physician assistant, nurse practitioner)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e25.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eAge\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e(median\u0026thinsp;=\u0026thinsp;49.0, IQR\u0026thinsp;=\u0026thinsp;25.0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e20\u0026ndash;29\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e25.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e30\u0026ndash;39\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e12.5\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e40\u0026ndash;49\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e12.5\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e50\u0026ndash;59\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e25.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e60 or more\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e25.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eYears of work experience\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e(median\u0026thinsp;=\u0026thinsp;11.0, IQR\u0026thinsp;=\u0026thinsp;15.9)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u0026ndash;14\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e50.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e15\u0026ndash;29\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e25.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e30 or more\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e25.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eInterviews\u003c/p\u003e\n\u003cp\u003eEight individual in-depth interviews were performed by author EB in weeks 18 through 22 of 2024. Although an interview guide was used (which is available as supplementary material to this paper), topics of interest could be pursued during the conversation in between the prepared questions.(\u003cspan class=\"CitationRef\"\u003e13\u003c/span\u003e) This fit the variation we anticipated in responses to the patient cases and the CKM. We preferred individual interviews to a focus group to allow each participant to speak freely and without being influenced by colleagues, so we might elicit any interprofessional differences. Audio recordings of the interviews were made to aid analysis and avoid the need for field notes.\u003c/p\u003e\n\u003cp\u003eThe interviews were divided into two parts. First, participants were sequentially presented with cases \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e\u0026ndash;3 from Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e. They were given time to study the patient\u0026rsquo;s data and were then asked to respond with the recommendations they would give this patient, as if it were a real patient. This open format was used to mimic a real-world scenario as closely as possible, which would not be possible if the participant needed to choose from prepared answers. Secondly, participants were presented with cases \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e\u0026ndash;6 together with the recommendations previously generated by the CKM. Instead of being asked to provide recommendations themselves, participants were to indicate whether they agreed with the recommendations and why.\u003c/p\u003e\n\u003ch2\u003eData analysis\u003c/h2\u003e\n\u003cp\u003eThe authors EB and EE inductively coded all relevant themes from the interview data, following Thomas\u0026rsquo; method.(\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e) Discrepancies in codes and themes were discussed between the coders to reach consensus.\u003c/p\u003e\n\u003cp\u003eThe interview data on cases \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e\u0026ndash;3 resulted in themes and responses concerning the clinical recommendations by the participants. To quantify any differences between the CKM-generated recommendations and that provided by our participants, we determined how often the major themes identified by the coding of the two sets of recommendations were not the same.\u003c/p\u003e\n\u003cp\u003eThe interview data on cases \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e\u0026ndash;6 produced themes on the participants\u0026rsquo; attitude towards the CKM. We also determined how often participants adopted the CKM\u0026rsquo;s recommendations or not and for what reasons.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cstrong\u003eCases 1-3: Themes regarding similarity of clinical recommendations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eConcerning the comparison between the recommendations given by participants and the CKM, the coding process produced six high-level themes and twenty-four related responses. These are displayed with frequencies in Table \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e. We will briefly discuss the main findings per theme.\u003c/p\u003e\n\u003cp\u003eTable 3: Frequency of all themes and responses extracted from the recommendations provided by participants and the CKM for synthetic patient cases 1-3. Calculated values are rounded to one decimal place. CKM-generated recommendations are marked with *.\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"609\" style=\"margin-right: calc(0%); width: 100%;\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 354px;\"\u003e\n \u003cp\u003e\u003cstrong\u003ethemes and responses\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eCase 1\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003ecase 2\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003ecase 3\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 354px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eRetake measurement\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;2 min\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;5 min\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;30 min\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;1 hour\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;After inhalation medication\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e7 (87.5%)\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e*\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003cp\u003e2 (25.0%)\u003c/p\u003e\n \u003cp\u003e4 (50.0%)\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e1 (12.5%)\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0 (0.0%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 354px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAdditional questions\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Fever\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Exertion\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Environmental and external factors (e.g. outside/inside,\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;daily routine, allergies, travelling abroad, contact with\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;animals)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Swelling or redness of leg\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Sputum appearance (e.g. color, consistency)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Deviation from normal symptoms\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Type of chest pain\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Onset of symptoms (e.g. gradual, sudden)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Urology symptoms (e.g. change in defecation, urinary\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;tract infection)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Weight\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Medical interventions (e.g. surgery, chemotherapy)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Smoking habit\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Cardiac history\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Patient\u0026rsquo;s assessment of their situation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e6 (75.0%)\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e3 (37.5%)\u003c/p\u003e\n \u003cp\u003e6 (75.0%)\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e8 (100.0%)\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e5 (62.5%)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e2 (25.0%)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e2 (25.0%)\u003c/p\u003e\n \u003cp\u003e5 (62.5%)\u003c/p\u003e\n \u003cp\u003e4 (50.0%)\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003cp\u003e3 (37.5%)\u003c/p\u003e\n \u003cp\u003e2 (25.0%)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e2 (25.0%)\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e6 (75.0%)\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e6 (75.0%)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e2 (25.0%)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e2 (25.0%)\u003c/p\u003e\n \u003cp\u003e4 (25.0%)\u003c/p\u003e\n \u003cp\u003e3 (37.5%)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e2 (25.0%)\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 354px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eContact with care professional\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Consultation by phone\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Check-up appointment\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Diagnostics (e.g. x-ray, MRI, lab tests)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e2 (25.0%)\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e2 (25.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e8 (100.0%)\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e5 (62.5%) *\u003c/p\u003e\n \u003cp\u003e1 (12.5%)\u003c/p\u003e\n \u003cp\u003e4 (50.0%) *\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e8 (100.0%)\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e4 (50.0%) *\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003cp\u003e6 (75.0%) *\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 354px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eMedication\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Increase inhaler dose\u003c/p\u003e\n \u003cp\u003e\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;Start new medication or antibiotic treatment\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0 (0.0%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e3 (37.5%)\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e1 (12.5%) *\u003c/p\u003e\n \u003cp\u003e2 (25.0%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e3 (37.5%)\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e*\u003c/p\u003e\n \u003cp\u003e3 (37.5)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 354px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eContinuation of monitoring\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e1 (12.5%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0 (0.0%) *\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0 (0.0%) *\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 354px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eLifestyle advice\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0 (0.0%)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0 (0.0%) *\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 85px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e0 (0.0%) *\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eRetake measurement\u003c/p\u003e\n\u003cp\u003eThis theme was most frequently mentioned in advice for case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e, with seven out of eight participants (87.5%) recommending retaking the measurement. Participants advised measuring the vitals again to verify if the initial measurement was correct, possibly taken after a physically straining activity. The responses included various suggested time intervals between the first and second measurements, ranging from 5 minutes to 1 hour. One participant recommended retaking the measurement after using inhalation medication. The model advised measuring the vitals again after 2 minutes, which was the shortest interval of all recommendations.\u003c/p\u003e\n\u003cp\u003eAdditional questions\u003c/p\u003e\n\u003cp\u003eThroughout all three cases in the first half of the interviews, many participants sought additional information about the patient.\u003c/p\u003e\n\u003cp\u003eWhen referring to environmental or external factors, participants meant any outside influence on the patient\u0026apos;s health. These factors included the location of the measurement (e.g. inside or outside), the patient\u0026rsquo;s daily routine, recent travel, contact with animals, allergy flare-ups, and exposure to sick individuals in their direct environment (e.g. at home). Deviations from normal symptoms were frequently inquired about because COPD patients often have certain consistent symptoms, such as shortness of breath during household activities. Understanding how current symptoms differ from usual ones can help gauge the severity of the situation. Finally, one participant asked for the patient\u0026rsquo;s self-assessment, inquiring if they could manage treatment at home while still taking care of themselves (e.g. eating and washing).\u003c/p\u003e\n\u003cp\u003eContact with care professional\u003c/p\u003e\n\u003cp\u003eThe recommendations by both the CKM and the participants revealed three responses related to contact with care professionals: phone consultations, (face-to-face) check-up appointments, and in-hospital diagnostics. In cases 2 and 3, all participants emphasized the importance of patient contact. For case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e, two respondents (25%) suggested a phone consultation to ask additional questions and assess speech and breathing patterns.\u003c/p\u003e\n\u003cp\u003eMedication\u003c/p\u003e\n\u003cp\u003eNone of the participants included medication in their advice for case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e, while three participants (37.5%) included it in cases 2 and 3. The CKM recommended increasing the existing inhaler dose for both these cases. However, nearly all the participants who mentioned medication in their advice preferred starting a new medication or antibiotic treatment instead.\u003c/p\u003e\n\u003cp\u003eContinuation of monitoring\u003c/p\u003e\n\u003cp\u003eWhile included by the CKM for both cases 2 and 3, only one participant included the continuation of monitoring in their advice for case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e. This involved asking the patient to re-enter their vital measurements and complete the questionnaires the next day to ensure the initial measurement was not erroneous and the patient was not unknowingly unwell.\u003c/p\u003e\n\u003cp\u003eLifestyle advice\u003c/p\u003e\n\u003cp\u003eThe model included lifestyle advice for both cases 2 and 3. However, none of the participants mentioned it. Lifestyle advice included recommendations such as using airway clearance techniques, practicing breathing and mindfulness exercises, and slowing down the pace of life.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCases 4-6: Themes regarding the level of agreement and attitude\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe found seven major themes and twenty-seven subthemes related to participants\u0026rsquo; level of agreement with the CKM-generated recommendations for cases \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e\u0026ndash;6 and their attitude towards the CKM. These are displayed in Fig. \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e.\u003c/p\u003e\n\u003cp\u003eWhile some themes appeared in both parts of the interviews, the responses differed due to the nature of the questions we asked. Actionable advice and reasons for adopting advice were not directly comparable.\u003c/p\u003e\n\u003cp\u003eMedication\u003c/p\u003e\n\u003cp\u003eParticipants considered administering medications, such as inhalers or antibiotics. In case \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e, most agreed to continue the current medication, with half of the participants adding that no additional medication was necessary. For case 5, those adopting the advice agreed on increasing the inhaler dose. Among non-adopters, opinions were divided: two participants supported increasing the dose, while the other two did not. The need for additional information and clarification on which inhaler to increase were the main reasons for non-adoption. Lastly, in case 6, most participants agreed on the necessity of increasing the inhaler dose, even though they did not adopt the advice.\u003c/p\u003e\n\u003cp\u003eMonitoring\u003c/p\u003e\n\u003cp\u003eThis theme was the most controversial, with responses reflecting differing opinions on the frequency of monitoring after entering a measurement in the CKM. The model suggested that patients should measure their vitals and fill in questionnaires the following day. Two participants recommended fewer measurements, adding that measurements only needed to be taken when symptoms arose. For case \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e, disagreements with the model\u0026rsquo;s advice on monitoring frequency were not perceived to be significant enough to prevent adoption of the overall advice.\u003c/p\u003e\n\u003cp\u003eSymptoms\u003c/p\u003e\n\u003cp\u003eThis theme focused on observations based on the vital measurements and questionnaire responses provided by the patient. Participants used these observations to justify their decisions. In case \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e, if vitals returned to normal and the patient felt well, no additional medication was deemed necessary. Conversely, worsening symptoms in case 5 might warrant an increased inhaler dose. Symptoms were less significant in case 6.\u003c/p\u003e\n\u003cp\u003eAdditional questions\u003c/p\u003e\n\u003cp\u003eSimilar to the first part of the interviews, participants sought additional information, though this theme was less prevalent in this second part. The responses were a subset of those identified during the first part. Despite its lower frequency, the theme significantly influenced responses. For example, in case 5, the need for additional information was important enough to reject the model\u0026rsquo;s advice.\u003c/p\u003e\n\u003cp\u003eContact with care professional\u003c/p\u003e\n\u003cp\u003eThis theme primarily addressed the role of the care professional in the treatment process. In case 6, the model recommended that patients initiate contact with their care professional and should undergo laboratory tests. However, participants disagreed with this approach, unanimously asserting that decisions regarding further treatment steps should be left to the care professional. While some participants agreed that laboratory tests should be performed, most suggested that it should not be included as recommendation to allow the treating professional to determine whether the tests are necessary or not.\u003c/p\u003e\n\u003cp\u003eLifestyle advice\u003c/p\u003e\n\u003cp\u003eThe responses varied similarly among participants regarding the lifestyle advice provided by the model in case 6, reflecting diverse perspectives. While a majority acknowledged the potential usefulness of breathing exercises, they generally perceived mindfulness exercises as less applicable. A few participants expressed concerns that patients experiencing anxiety from an exacerbation might find it challenging to engage effectively with mindfulness exercises.\u003c/p\u003e\n\u003cp\u003eQualities of CKM-generated advice\u003c/p\u003e\n\u003cp\u003eThe final major theme identified pertained to the qualities participants\u0026rsquo; attributed to the CKM-generated recommendations. Three responses were identified: unclear advice, excessive information in the advice, and potential confusion for the patient. There was consensus among participants, particularly evident in case 6, that the advice contained an excessive amount of information, potentially leading to confusion among patients. As previously mentioned, participants emphasized that if patients are advised to contact their care professional, subsequent treatment decisions should be left to the professional\u0026rsquo;s discretion. This concern was a primary factor influencing participants\u0026rsquo; decision not to adopt the advice in case 6.\u003c/p\u003e\n\u003cp\u003eKnowledge elicitation from expert versus CKM recommendations\u003c/p\u003e\n\u003cp\u003eThe difference between the CKM-generated recommendations and those expressed by participants was quantified for cases \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e\u0026ndash;3 by counting the number of deviations between the major themes. Deviations included instances where the participant either omitted advice recommended by the model or added advice not contained in the model\u0026rsquo;s recommendations. The total difference score is a combination of all participants\u0026rsquo; difference scores to allow for case-by-case comparison. Case 2 had the highest total difference score (29) and mean difference score (3.6 SD 0.7), indicating the greatest divergence from the model\u0026rsquo;s recommendations.\u003c/p\u003e\n\u003cp\u003eThe number of differences varied per case and theme. Across all three cases, participants asked additional questions in almost every interview (20 of 24 cases, 83.3%). Continuation of monitoring was absent in the participants\u0026rsquo; advice for cases 2 and 3, but was included once in case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e. Similarly, lifestyle advice was not present in the participants\u0026rsquo; recommendations for cases 2 and 3. Medication was included by three participants in cases 2 and 3, but never for case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e. The themes with the fewest deviations from the model\u0026rsquo;s advice were \u003cem\u003eretake measurement\u003c/em\u003e and \u003cem\u003econtact with care professional\u003c/em\u003e, each with only two differences. Retaking the measurement was consistently included in case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e when recommended by the model. Contact with the care professional was always advised in cases 2 and 3 when the model recommended it. A comprehensive overview of the differences per major theme, the absolute deviation score, and mean deviation score, is presented in Table \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e.\u003c/p\u003e\n\u003cp\u003e\u003c/p\u003e\n\u003ctable id=\"Tab4\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eDifferences per major theme, split by case and in total. For each case, the left column shows the frequency of each theme as mentioned by participants (P). The right column compares the frequency of these themes in the participants\u0026rsquo; recommendations with the CKM\u0026rsquo;s recommendations. A positive number indicates the theme was present in the participants\u0026rsquo; recommendations but not in the CKM\u0026rsquo;s, while a negative number indicates the theme was in the CKM\u0026rsquo;s recommendations but not in the participants\u0026rsquo;. The total deviation score is the absolute sum of these differences. The mean deviation score is the total absolute deviation score divided by the number of participants. Calculated values are rounded to one decimal place. CKM-generated recommendations are marked with *.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\u0026nbsp;\u003c/th\u003e\n \u003cth align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eCase \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003ecase 2\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003ecase 3\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\u0026nbsp;\u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ethemes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003ePs\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e+/-\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003ePs\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e+/-\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003ePs\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e+/-\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eTotal difference score\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eRetake measurement\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7 *\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e+\u0026thinsp;1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAdditional questions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e+\u0026thinsp;6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e+\u0026thinsp;8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e+\u0026thinsp;6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e20\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eContact with care professional\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e+\u0026thinsp;2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8 *\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e8 *\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMedication\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3 *\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3 *\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eContinuation of\u003c/p\u003e\n \u003cp\u003emonitoring\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e+\u0026thinsp;1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0 *\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e17\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLifestyle advice\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0 *\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e-8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e16\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAbsolute difference\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e10\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e30\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003e27\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMean difference,\u003c/p\u003e\n \u003cp\u003estandard deviation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1.3\u003c/p\u003e\n \u003cp\u003eSD 0.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.6\u003c/p\u003e\n \u003cp\u003eSD 0.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.4\u003c/p\u003e\n \u003cp\u003eSD 0.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003eInterprofessional differences\u003c/p\u003e\n\u003cp\u003eThe second aim of this study was to examine potential interprofessional differences in treatment strategies among the participating care professionals. This was also done using difference scores. Figure \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e illustrates the absolute number of differences between the recommendations of the participants in the form of a heatmap, with darker colors indicating higher numbers of differences between the responses in the recommendations. The average number of differences per case was 3.4 (SD 1.3) for case \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e, 7.3 (SD 2.2) for case 2, and 5.6 (SD 2.2) for case 3, all rounded to one decimal place.\u003c/p\u003e\n\u003cp\u003eThe average number of interprofessional differences between and within professional groups is presented in Table \u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e. For pulmonologists, the number of interprofessional clinical differences within the group was lower than the differences between pulmonologists and the other groups. Conversely, for pulmonologists in training and non-physicians, the differences within the group were higher than between groups. Notably, among non-physicians, the number of within-group differences was the highest.\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cdiv class=\"gridtable\"\u003e\u0026nbsp;\u003ctable id=\"Tab5\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 5\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eAverage number of differences between the three groups of care professionals (pulmonologists, pulmonologists in training, and non-physicians) across cases \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e\u0026ndash;3. The diagonal presents the average number of differences between the care professionals within the same group. Calculated values are rounded to one decimal place.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\u0026nbsp;\u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003ePulmonologists\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003ePulmonologists in training\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eNon-physician with patient contact\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePulmonologists\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e12.7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e16.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e16.3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePulmonologists in training (resident)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e16.8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e18.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e16.3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNon-physician with patient contact\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e16.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e16.3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e19.0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003eAdoption of CKM-generated recommendations\u003c/p\u003e\n\u003cp\u003eThe adoption of the model\u0026rsquo;s advice varied across cases \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e\u0026ndash;6. As shown in Fig. \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e, all care professionals adopted the advice for case \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e, despite disagreements about the recommended frequency of monitoring. For case 5, adoption was split evenly, with half of the care professionals agreeing with the model\u0026rsquo;s advice and the other half disagreeing, primarily due to differing opinions on medication administration. In case 6, nearly all participants rejected the model\u0026rsquo;s advice, except for one. They criticized the advice for containing too much information, which could confuse the patient or complicate the care professional\u0026rsquo;s decision-making process. One professional noted, \u0026ldquo;\u003cem\u003eIf you say \u0026lsquo;contact your care professional\u0026rsquo;, the rest of the treatment plan should be left up to that care professional.\u003c/em\u003e\u0026rdquo; Recommending laboratory tests could limit the care professional\u0026rsquo;s ability to consider alternative actions, as patients might assume lab tests are essential for proper treatment.\u003c/p\u003e\n\u003cp\u003eParticipants with the same professional role tended to accept the same cases. While all the pulmonologists accepted the model\u0026rsquo;s advice for case 5, both of the other groups of professionals (pulmonologists in training and non-physicians with patient contact) did not accept the advice. The acceptance behavior for the pulmonologists in training and the non-physicians was identical, both within and between the groups. The pulmonologists had clinical differences both within their group, as well as between them and the other two groups.\u003c/p\u003e\n\u003cp\u003eFigure \u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e displays how acceptance of CKM-generated recommendations increased with the age of participants.\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eThe concept of decision support for the (self-)management of COPD exacerbations is well-established, with several alternatives to the COPD CKM having been developed.(\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e, \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e) The validation of these tools, however, varies significantly. The COPDPredict\u0026trade; application, for example, was validated by demonstrating its usability for both patients and physicians, and by showing that its prediction algorithm can reliably inform both user groups about acute events.(\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e) This was achieved by comparing \u0026ldquo;algorithm-defined\u0026rdquo; exacerbations to \u0026ldquo;clinician-defined\u0026rdquo; exacerbations. As this validation method was established to properly validate a COPD monitoring tool, our study applied a similar approach. It compared the automated output of the COPD clinical knowledge model to human-generated output to ensure the model aligns with good clinical practice. This approach helps prevent a decline in care quality as healthcare becomes more automated. In contrast, the ACCESS prediction model was validated using statistical methods, such as an ROC-analysis.(\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e) Although the model was developed with input from pulmonologists, its evaluation did not include validation with other pulmonary specialists. A later evaluation of the ACCESS software focused on the validity of its most important treatment advice (contact the healthcare professional).(\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e)\u003c/p\u003e \u003cp\u003eOur current study evaluated the entire advice rather than just a subset. The results provide insight into possible areas of domain knowledge that require further elicitation to expand the usability of the CKM. They also highlight interprofessional clinical differences, and demonstrate the rate of adoption among care professionals based on a variety of characteristics. To the authors\u0026rsquo; knowledge, this is the first study to evaluate the human versus CKM performance in COPD monitoring based on synthetic patient data.\u003c/p\u003e \u003cp\u003eA principal finding of this study was the significant variation between the model\u0026rsquo;s and care professionals\u0026rsquo; recommendations across all three cases. Overall, the participants omitted more than half of the responses included in the model\u0026rsquo;s advice. At the same time, they consistently sought more patient information, particularly regarding body temperature, exertion, and sputum appearance. One possible explanation is that the participants felt they needed more information to thoroughly understand their patient\u0026rsquo;s status. Another possibility is that care professionals are accustomed to different ways of collecting information and taking decisions. If so, the CKM\u0026rsquo;s current presentation might not fit these habits. Further user-centered research may shed light on whether adjustments to the CKM and/or the information collecting and processing habits of care professionals might be beneficial for their collaboration.\u003c/p\u003e \u003cp\u003eParticipants also tended to prescribe more impactful treatments than the model recommended. While the model suggested increasing the inhaler dose, some participants preferred starting antibiotic treatments, indicating that the model\u0026rsquo;s recommendations were perceived to be insufficient for their patient\u0026rsquo;s needs.\u003c/p\u003e \u003cp\u003eBoth the need for additional information and the treatment manner suggest that the care professionals took a more cautious approach than the model when treating a patient. While a cautious approach is not necessarily problematic, it could lead to overtreatment. In recent work, it has been found that overtreatment of COPD occurs in almost one in two cases, meaning that this is a prevalent issue within COPD care.(\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e) The CKM could prevent overtreatment by giving physicians the confidence to work with less information and to only seek additional information when it is required for decision making. At the same time, it may also prompt the use of information that is currently often neglected, such as spirometry in the diagnostic phase of COPD. This is a piece of information with high specificity that can prevent the erroneous diagnose of COPD, but is not always considered.(\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e)\u003c/p\u003e \u003cp\u003eOverall, the rich results of comparing the expert recommendations with the CKM\u0026rsquo;s indicate that case-based interviewing is a suitable technique for knowledge elicitation regarding clinical knowledge. This is important, because CKMs and their performance are entirely based on available domain knowledge. In this, they are distinct from \u003cem\u003edata-driven\u003c/em\u003e solutions, such as machine learning, in which new knowledge is encoded in the model by having it \u0026ldquo;learn\u0026rdquo; from large quantities of data.(\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e) Such data-driven knowledge has its place in medicine, for example in predictive models and image processing, but can be unsatisfying for healthcare professionals who require more explainability of model-generated recommendations, either for themselves or for their patients.(\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e) CKMs allow tracing each recommendation back to all the rules and inputs that led to the recommendation being generated. A disadvantage of this approach for CKMs is that they are unable to generate new rules based on available data. A possible architecture to mitigate this would be a two-agent system, in which the CKM is complemented by a second data-driven component. This second component might generate new rules through induction, by observing the recommendations from the CKM, the registered actions taken by healthcare professionals, and patient outcomes, forming a closed-loop system.\u003c/p\u003e \u003cp\u003eThe elicited knowledge revealed evident interprofessional clinical differences between different types of care professionals and within groups of the same type. This forms a significant hurdle in expanding the accepted knowledge encoded in the CKM. To the authors\u0026rsquo; knowledge, no prior studies have specifically examined differences among pulmonary care professionals treating COPD patients. A recent publication by Svensson and Jacobsson highlighted inconsistencies in decision-making among emergency department care professionals.(\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e) Their framework identified three modes of medical decision-making: experiential-based (emotional and intuitive), ostensive-based (rational and rule-based), and action-based (physician\u0026rsquo;s doings in the situation). The interplay between these decision-making modes varies between care professionals, leading to an eight-fold typology of medical decision-making praxis. This suggests that interprofessional differences in medical decision-making are not unique to the pulmonary specialists included in this study and warrants further research to identify the causes of these discrepancies. Additionally, our method resulted in material that the participants and their colleagues can use to further discuss their professional differences. This is valuable, even outside of their possible continued use of the CKM.\u003c/p\u003e \u003cp\u003eThe third objective of this research was to examine the adoption behavior of care professionals and whether characteristics such as job experience and age influence this behavior. Overall, the proposed advice from the model was adopted by participants about half of the time, indicating a need for further consultation with more professionals during model development to achieve greater consensus.\u003c/p\u003e \u003cp\u003eBoth characteristics \u0026ndash; job experience and age \u0026ndash; impacted the adoption of the model\u0026rsquo;s advice. The trend that the experienced professionals accepted the model\u0026rsquo;s advice more readily than other groups contrast with prior research. Adler-Milstein and colleagues have suggested that experienced professionals might distrust AI tools, especially if the recommendations contradict their intuitive conclusions.(\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e) Similarly, studies on automation bias indicate that less experienced professionals are more prone to over-relying on automation, suggesting that experienced professionals would be less likely to adopt AI-generated recommendations.(\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e, \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e) The contrast between this study\u0026rsquo;s findings and existing literature highlights the need for further research to understand the correlation between a care professional\u0026rsquo;s experience and the adoption of automatically generated recommendations.\u003c/p\u003e \u003cp\u003eGiven that the more experienced participants were older, it is challenging to determine whether experience or age has a greater effect on adoption rates. Nonetheless, the results suggest that adoption rates increase with age. This contradicts several studies suggesting that younger individuals are more likely to adopt AI tools due to their generally techno-positive attitude, which aligns with the idea that the impact of age decreases as experience increases.(\u003cspan additionalcitationids=\"CR27\" citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e) A larger and more varied study population would be required to conclusively determine whether age is correlated with the adoption rate of automatically generated recommendations.\u003c/p\u003e \u003cp\u003eFinally, some adoption behaviors showed consensus among professionals. For instance, nearly all care professionals rejected the advice for case 6 due to excessive information. Conversely, a theme that caused discussion but did not influence the adoption rate was the frequency of monitoring. This suggests that as long as the recommendations appear sensible, care professionals accept a certain degree of interprofessional differences in the specific execution of the advice.\u003c/p\u003e \u003cp\u003eA significant strength of this study is its inclusion of the intended user group in the evaluation process. By involving care professionals who will use the CKM, the study facilitates their gradual familiarization with the tool before its implementation. Additionally, involving care professionals in the tool\u0026rsquo;s development fosters a sense of responsibility for its outcome and implementation, potentially increasing its adoption when it is eventually ready to be integrated into COPD care. Patients were intentionally excluded from this preliminary evaluation of the CKM to first determine if the model aligned with current medical practice. Future studies should include both patients and care professionals to assess the tool\u0026rsquo;s usability.\u003c/p\u003e"},{"header":"Conclusions","content":"\u003cp\u003eCase-based interviews proved an effective method for eliciting clinical knowledge required for completing a CKM. Themes that emerged included retaking measurements, asking the patient additional questions, contacting the care professional, medication, continuation of monitoring, and lifestyle advice. In many of their responses, participants exhibited more caution than the CKM, seeking additional information and proposing stronger measures, such as medication.\u003c/p\u003e \u003cp\u003eInterprofessional differences were found between different types of care professionals, as well as within groups of the same type. Pulmonologists were most unanimous in their advice, while variations between all three groups were similar.\u003c/p\u003e \u003cp\u003eThe care professionals showed a moderate willingness to adopt the CKM, with the proposed recommendations accepted slightly more than half the time. This suggests that further collaboration with care professionals is necessary to develop advice that gains wider acceptance. Higher adoption rates were associated with older, more experienced professionals, although the interplay between these characteristics and their precise influence on adoption rates remains inconclusive.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eCERT\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eCOPD Exacerbation Recognition Tool\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eCKM\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eclinical knowledge model\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eCOPD\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003echronic obstructive pulmonary disease\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eEXACT\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eExacerbations of Chronic Pulmonary Disease Tool\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eMUMC+\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eMaastricht University Medical Center+\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eSD\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003estandard deviation\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003ch2\u003eEthics approval and consent to participate\u003c/h2\u003e\n\u003cp\u003eThis study was conducted in accordance with the ethical principles of the Netherlands Code of Conduct for Research Integrity (2018). All participants signed an informed consent form agreeing with participating in the research described in this paper. All data collected was pseudonymized, with only the author who conducted the interviews having the key. All recordings of interviews were destroyed after analysis. The Research Ethics Committee of the Faculty of Health, Medicine and Life Sciences of Maastricht University approved the research described in this paper under case number FHML/HDT/2024.002.\u003c/p\u003e\n\u003ch2\u003eConsent for publication\u003c/h2\u003e\n\u003cp\u003eAll participants signed an informed consent form agreeing with the publication of their data in this paper.\u003c/p\u003e\n\u003ch2\u003eAvailability of data and materials\u003c/h2\u003e\n\u003cp\u003eThe interview data are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003ch2\u003eCompeting interests\u003c/h2\u003e\n\u003cp\u003eFT, PRM, WB, and HtB are shareholders of Open Walnoot, a company that specializes in providing the techniques and solutions described in this paper to various clients within the healthcare sector.\u003c/p\u003e\n\u003ch2\u003eFunding\u003c/h2\u003e\n\u003cp\u003eNo supplemental funding was received for the execution of this research project.\u003c/p\u003e\n\u003ch2\u003eAuthor’s contributions\u003c/h2\u003e\n\u003cp\u003eEB and EE executed the majority of the research and drafted the manuscript, with significant support from SS, MS, and FvM. FT, PRM, WBa, and HtB constructed the clinical knowledge model in collaboration with SS. FT, PRM, WBa, and HtB all contributed equally in sharing their relevant materials, experience and insights. All authors provided additions and modifications to the draft versions of the manuscript.\u003c/p\u003e\n\u003ch2\u003eAcknowledgements\u003c/h2\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eEdelman E, Tijssen F, Munniksma PR, Bast W, Bohmer H, van ten, Eldik N et al Clinical knowledge modeling: An essential step in the digital transformation of healthcare. The Innovation [Internet]. 2024 Nov 4 [cited 2025 Feb 28];5(6). Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.cell.com/the-innovation/abstract/S2666-6758(24)00156-5\u003c/span\u003e\u003cspan address=\"https://www.cell.com/the-innovation/abstract/S2666-6758(24)00156-5\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKlemann D, Winasti W, Tournois F, Mertens H, van Merode F (2024) Quantifying the Resilience of a Healthcare System: Entropy and Network Science Perspectives. Entropy 26(1):21\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGalbraith JR (1974) Organization Design: An Information Processing View. Interfaces 4(3):28\u0026ndash;36\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePatel VL, Arocha JF, Kaufman DR (1999) Expertise and Tacit Knowledge in Medicine. Tacit Knowledge in Professional Practice. Psychology\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWickramasinghe N, Davison G (2004) Making Explicit the Implicit Knowledge Assets in Healthcare: The Case of Multidisciplinary Teams in Care and Cure Environments. Health Care Manag Sci 7(3):185\u0026ndash;195\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTing SL, Wang WM, Tse YK, Ip WH (2011) Knowledge elicitation approach in enhancing tacit knowledge sharing. Ind Manag Data Syst 111(7):1039\u0026ndash;1064\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ede Gans ST, Maessen GC, van de Pol MHJ, van Apeldoorn MJ, van Ingen-Stokbroekx MAL, van der Sloot N et al (2023) Effect of interprofessional and intraprofessional clinical collaboration on patient related outcomes in multimorbid older patients \u0026ndash; a retrospective cohort study on the Intensive Collaboration Ward. BMC Geriatr 23(1):1\u0026ndash;11\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJones PW, Wang,Chanzheng C et al (2022),Ping, Chen, Liping, Wang, Daoxin, Xia, Junbo,. The Development of a COPD Exacerbation Recognition Tool (CERT) to Help Patients Recognize When to Seek Medical Advice. Int J Chron Obstruct Pulmon Dis. ;17:213\u0026ndash;22\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLeidy NK, Wilcox TK, Jones PW, Murray L, Winnette R, Howard K et al (2010) Development of the EXAcerbations of Chronic Obstructive Pulmonary Disease Tool (EXACT): A Patient-Reported Outcome (PRO) Measure. Value Health 13(8):965\u0026ndash;975\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRaghunathan TE (2021) Synthetic Data. Annu Rev Stat Its Appl. ;8(Volume 8, 2021):129\u0026ndash;40\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMcLachlan S Realism in synthetic data generation\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMurtaza H, Ahmed M, Khan NF, Murtaza G, Zafar S, Bano A (2023) Synthetic data generation: State of the art in health care domain. Comput Sci Rev 48:100546\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eQu SQ, Dumay J (2011) The qualitative research interview. Qual Res Acc Manag 8(3):238\u0026ndash;264\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eThomas DR (2006) A General Inductive Approach for Analyzing Qualitative Evaluation Data. Am J Eval 27(2):237\u0026ndash;246\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003evan der Heijden M, Lucas PJF, Lijnse B, Heijdra YF, Schermer TRJ (2013) An autonomous mobile system for the management of COPD. J Biomed Inf 46(3):458\u0026ndash;469\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePatel N, Kinmond,Kathryn, Jones, Pauline, Birks, Pamela, and, Spiteri MA (2021) Validation of COPDPredict\u003csup\u003e\u0026trade;\u003c/sup\u003e: Unique Combination of Remote Monitoring and Exacerbation Prediction to Support Preventative Management of COPD Exacerbations. Int J Chron Obstruct Pulmon Dis. ;16:1887\u0026ndash;99\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBoer LM, van der Heijden ME, Lucas, Peter JF, Vercoulen, Jan H et al (2018) Assendelft, Willem JJ,. Validation of ACCESS: an automated tool to support self-management of COPD exacerbations. Int J Chron Obstruct Pulmon Dis. ;13:3255\u0026ndash;67\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFiore M, Ricci M, Rosso A, Flacco ME, Manzoli L (2023) Chronic Obstructive Pulmonary Disease Overdiagnosis and Overtreatment: A Meta-Analysis. J Clin Med 12(22):6978\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHeffler E, Crimi C, Mancuso S, Campisi R, Puggioni F, Brussino L et al (2018) Misdiagnosis of asthma and COPD and underuse of spirometry in primary care unselected patients. Respir Med 142:48\u0026ndash;52\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDubois D, H\u0026aacute;jek P, Prade H (2000) Knowledge-Driven versus Data-Driven Logics. J Log Lang Inf 9(1):65\u0026ndash;89\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eReddy S (2022) Explainability and artificial intelligence in medicine. Lancet Digit Health 4(4):e214\u0026ndash;e215\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSvensson M, Jacobsson M (2024) Managing inconsistencies in medical decision-making: An eight-fold typology. Eur Manag J 42(1):130\u0026ndash;141\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAdler-Milstein J, Aggarwal N, Ahmed M, Castner J, Evans BJ, Gonzalez AA et al (2022) Meeting the Moment: Addressing Barriers and Facilitating Clinical Adoption of Artificial Intelligence in Medical Diagnosis. NAM Perspect. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.31478/202209c\u003c/span\u003e\u003cspan address=\"10.31478/202209c\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDratsch T, Chen X, Rezazade Mehrizi M, Kloeckner R, M\u0026auml;hringer-Kunz A, P\u0026uuml;sken M et al (2023) Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance. Radiology 307(4):e222176\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGoddard K, Roudsari A, Wyatt JC (2014) Automation bias: empirical results assessing influencing factors. Int J Med Inf 83(5):368\u0026ndash;375\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuisman M, Ranschaert E, Parker W, Mastrodicasa D, Koci M, Pinto de Santos D et al (2021) An international survey on AI in radiology in 1,041 radiologists and radiology residents part 1: fear of replacement, knowledge, and attitude. Eur Radiol 31(9):7058\u0026ndash;7066\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKhanijahani A, Iezadi S, Dudley S, Goettler M, Kroetsch P, Wise J (2022) Organizational, professional, and patient characteristics associated with artificial intelligence adoption in healthcare: A systematic review. Health Policy Technol 11(1):100602\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRushlow DR, Croghan IT, Inselman JW, Thacher TD, Friedman PA, Yao X et al (2022) Clinician Adoption of an Artificial Intelligence Algorithm to Detect Left Ventricular Systolic Dysfunction in Primary Care. Mayo Clin Proc. ;97(11):2076\u0026ndash;85\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-digital-health","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [BMC Digital Health](https://bmcdigitalhealth.biomedcentral.com/)","snPcode":"44247","submissionUrl":"https://submission.nature.com/new-submission/44247/3","title":"BMC Digital Health","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"clinical knowledge model, knowledge engineering, chronic obstructive pulmonary disease, remote patient monitoring, automation, decision support","lastPublishedDoi":"10.21203/rs.3.rs-6819489/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6819489/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eEncoding medical knowledge in a digital clinical knowledge model (CKM) enables its usage for automation and decision support. Formalized sources of knowledge are usually not sufficient to construct a complete model. Clinical experts may also hold relevant, implicit knowledge that they have gained through experience or from other unrecorded sources. Our aim was to study whether early modeling of medical knowledge in a CKM and the fictional application of the resulting model together with clinical experts might help elicit this semi-hidden, but highly relevant domain knowledge.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe created a CKM to support patients suffering from chronic obstructive pulmonary disease in self-managing their condition by generating recommendations based on measurements and questionnaires they perform at home. We subsequently interviewed 8 pulmonary experts about their recommendations for synthetic patient cases versus those generated by the CKM. At the same time, we collected feedback from the professionals to study their attitude towards the CKM and its generated recommendations.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe interviews enabled us to elicit further domain knowledge on various themes: retaking measurements, asking the patient additional questions, contacting the care professional, medication, continuation of monitoring, and lifestyle advice. Secondly, the elicited knowledge revealed interprofessional differences between different types of care professionals and within groups of the same type. Additionally, our results show a trend that the experienced professionals accepted the model’s advice more readily than other groups.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe themes we identified indicate that case-based interviewing is a suitable technique for knowledge elicitation regarding clinical knowledge. The interprofessional differences in recommendations form a hurdle in expanding the accepted knowledge encoded in the CKM. The experienced professionals being more accepting of the model’s advice contrasts with existing literature. This highlights the need for further research to understand the correlation between a care professional’s experience and the adoption of automatically generated recommendations.\u003c/p\u003e\n\u003cp\u003ePatients were intentionally excluded from this preliminary evaluation of the CKM to first determine if the model aligned with current medical practice. Future studies should include both patients and care professionals to assess the tool’s usability.\u003c/p\u003e","manuscriptTitle":"Capturing Clinical Knowledge: The Digital Modeling of Expert Knowledge Concerning the Care for Patients with Chronic Obstructive Pulmonary Disease","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-06-18 00:21:45","doi":"10.21203/rs.3.rs-6819489/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-03-28T07:21:02+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-27T19:47:26+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"269878587119462394205691803901137145042","date":"2026-03-06T22:50:33+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"146856621143024268930068758136813608663","date":"2026-02-01T23:25:28+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-07-06T07:42:15+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"36709760792466546912986399650365621076","date":"2025-06-12T08:26:59+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-06-12T02:04:49+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-06-12T01:55:27+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-06-09T13:10:44+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-06-06T15:42:59+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Digital Health","date":"2025-06-06T15:40:10+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-digital-health","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [BMC Digital Health](https://bmcdigitalhealth.biomedcentral.com/)","snPcode":"44247","submissionUrl":"https://submission.nature.com/new-submission/44247/3","title":"BMC Digital Health","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"47046624-b3f4-4040-8e4d-15c909c255b0","owner":[],"postedDate":"June 18th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-05-22T10:09:51+00:00","versionOfRecord":[],"versionCreatedAt":"2025-06-18 00:21:45","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6819489","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6819489","identity":"rs-6819489","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}