The impact of a targeted education package on the knowledge, attitudes, and utilisation of patient reported outcome measures amongst chiropractors in Australia.

The impact of a targeted education package on the knowledge, attitudes, and utilisation of patient reported outcome measures amongst chiropractors in Australia. | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article The impact of a targeted education package on the knowledge, attitudes, and utilisation of patient reported outcome measures amongst chiropractors in Australia. Natalie Clohesy, Anthony Schneiders, Gaery Barbery, Steven Obst This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1907999/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 6 You are reading this latest preprint version Abstract Background: Patient Reported Outcome Measures (PROMs) have been shown to be valid and reliable indicators of health status and treatment outcomes, however, the current knowledge, understanding, and utilisation of PROMs within the Australian Chiropractic profession is limited. This study used the New World Kirkpatrick Model (NWKM) to evaluate whether an online PROM education package could improve knowledge, confidence, attitude, and utilisation of PROMs by chiropractors in Australia. Methods: A longitudinal cohort interventional study of chiropractors in Australia. The recruitment phase and data collection period occurred from November 2020 – May 2021. Participants completed three online surveys two weeks before, four weeks after, and 12 weeks after receiving an online education package that included ten evidence-based region-specific modules on PROMs. Survey questions were grouped into five subthemes for analysis according to the NWKM levels: 1) Reaction; 2) Learning - knowledge; 3) Learning – confidence; 4) Behaviour – attitude; 5) and Behaviour - utilisation). Results: Of the 113 participants that enrolled in the study, 43 completed all three survey and were included in the analysis. There was very positive to the education package with mean response scores (1-5 Likert scale) for the reaction questions ranging from 3.75 to 4.43. There was a small, but significant, increase in knowledge (out of 32) at four weeks (24.3 ± 6.1) and 12 weeks after receiving the education package (27.2 ± 5.5), compared to baseline (27.4 ± 5.1). There was no effect of intervention on clinician confidence or attitude towards PROMs. Utilisation of function- and pain-related PROMs did not change after the intervention. There was a small and significant (p<0.05) increase in utilisation of health-related PROMs 12 weeks after the intervention. Conclusion: Despite modest improvements in knowledge, which were retained 12 weeks after the educational package was provided, there was no evidence that participant confidence, attitude, or utilisation of PROMs changed because of the intervention. While the respondents’ have positive attitudes and beliefs regarding PROMs use, further education surrounding the clinical translation process into clinical practice is required. Patient reported outcome measures Chiropractic Knowledge to Action New World Kirkpatrick Model Behaviour change Background Patient reported outcome measures (PROMs) are validated questionnaires and survey instruments used to measure the status of a patient’s health condition using information that comes directly from the patient [ 1 , 2 ]. PROMs were initially used as epidemiological surveys to identify patterns of symptomatology or health status, however, they have evolved to be tools that can also be used to support clinical practice and research [ 3 ]. In the clinical setting, PROMs can assist the decision-making process, improve therapist-patient communication, monitor treatment progress, and/or facilitate patient-centered care [ 4 – 7 ]. PROMs are therefore considered an integral part of evidence-based health care and therefore it is important that clinicians are equipped with the requisite skills and knowledge needed to apply PROMs in clinical practice. A recent survey of chiropractors in Australia found that 72% of respondents reported using PROMs for the management of low back pain (LBP) [ 8 ]. Similar rates of PROM utilisation were reported for physiotherapists in the United Kingdom, where 72% of physiotherapists, and 71% of physiotherapy practices, routinely used PROMs [ 9 ]. Despite these relatively high rates, it has been suggested there is a lack of knowledge and understanding of PROMs amongst chiropractors, which may limit the full clinical implementation of PROMs and their potential benefits [ 5 , 8 – 11 ]. For example, a survey of chiropractors in Australia found that 19% of respondents reported that a lack of knowledge regarding PROMs prevented clinical implementation, while time required to administer and score PROMs in a clinical setting (17%) was also a barrier to their utilisation [ 8 ]. The results of studies by Holmes, Bishop [ 5 ] and Antunes, Harding [ 12 ] mirrored these findings, suggesting that a clinician’s lack of PROM knowledge was a significant barrier to implementation. Although chiropractors are increasingly encouraged to use PROMs [ 13 ], a lack of knowledge transfer from literature, education, and training into clinical practice are likely barriers to their successful implementation into clinical practice [ 5 , 14 ]. Furthermore, even though chiropractors place high value on the importance of PROMs in clinical practice, these beliefs do not appear to translate to increased frequency of PROM usage, suggesting other factors, including knowledge, may be central to changing behaviors [ 15 ]. To improve the transfer of knowledge to practice, a number of theories, models and frameworks have been developed [ 16 ]. Within allied health settings, the most highly cited and accepted model is the Knowledge to Action framework (KTA) [ 17 – 20 ] created by Graham and Tetroe [ 21 ]. KTA is a conceptual framework made up of two distinct components - knowledge creation and the action cycle. The knowledge creation phase is represented as a tunnel and the information becomes more refined and specific as it passes through this phase to uncover the most valid and useful knowledge [ 22 ]. The action phase is a cycle leading to implementation or application of knowledge uncovered in the knowledge creation phase. During this step, adapting knowledge to meet the needs within particular contexts or populations occurs [ 23 ]. The aim of this study was to evaluate the impact of an educational package developed and implemented using KTA framework to improve PROM knowledge and utilisation in a cohort of chiropractors in Australia. We hypothesised that a tailored educational package intervention would increase participant knowledge of PROMs, which would translate to a positive change in attitude and frequency of PROM utilisation in clinical practice. Methods Study Design A longitudinal interventional cohort survey with repeated measures was used to gain an understanding of the respondent’s knowledge, attitudes, and utilisation of PROMs in the Chiropractic profession of Australia. An online educational package was designed, delivered and the findings evaluated using the New World Kirkpatrick Model (NWKM) [ 24 – 26 ] using three of the four key themes of the model: reaction, learning, and behaviour. The study was conducted over 7 months and included three online surveys administered at specific timepoints using the Qualtrics software. Full ethical approval was granted by the Human Research Ethics Committee of Central Queensland University, reference number 0000022391. Participants Chiropractors who were current members of the Australian Chiropractic Association (ACA) or Chiropractic Australia (CA) were invited to participate in the study via member emails and advertisement in the association’s newsletters. The sample population consisted of 3,215 ACA members and 1,163 CA members, totaling 4,378 potential participants, representing the majority of Chiropractors registered and practicing in Australia in 2020 [ 27 ]. The recruitment phase and data collection period occurred from November 2020 – May 2021. Longitudinal Surveys Participants were surveyed on three occasions: 1) two weeks before the educational package (Survey 1); 2) four weeks after receiving the educational package (Survey 2); and 3) 12 weeks after receiving the educational package (Survey 3). Survey 1 collected demographic information including age, sex, country of graduation, highest level of education, in addition to the participant’s knowledge and attitudes about PROMs and their preferred mode of delivery (e.g., audio, audio-visual or transcript) for the educational package. Survey 2 and 3 repeated the knowledge and attitude questions from Survey 1, while omitting the demographic questions, and including items assessing the participant’s reaction to the educational package. Educational Package The content and design of the educational package was informed by data obtained from Survey 1, and therefore, being co-designed [ 28 ]. Respondents were given multiple-choice questions with options of content to be included in the educational package. There was also an open text box to allow them to formulate their own answers and request specific information to be included, which is recommended when using survey methods [ 29 ]. Survey 1 also asked the respondents about their preference of mode for the delivery of the educational package. Of the 116 respondents, 81% were very likely/likely to view via text (electronic), 57% via audio-video format, 55% via text (paper), and 35% via audio. Given the spread of preferred formats the educational package was adapted and made available via for all three modes – audio-visual, audio and text [ 30 ]. When asked about what topics should be included in the package, most respondents requested ‘Examples of the best PROMs for major conditions’ (85%), ‘How PROMs can be used in Chiropractic clinical practice (83%)’, and ‘Explanation of categories of PROMs (80%)’. The subsequent educational package consisted of 10 modules: 1) introduction; 2) background of PROMs; 3) PROMs in Chiropractic practice; 4) non-location specific PROMs; 5) PROMs for headache, whiplash, temporomandibular joint and dizziness; 6) PROMs for cervical spine; 7) PROMs for thoracic spine; 8) PROMs for lumbar spine; 9) PROMs for upper extremity; and 10) PROMs for lower extremity. For modules 4–10, the following topics were included (where available and/or appropriate) for each example PROM: category, overview, scoring, area of assessment, method of delivery, and clinical condition(s). The duration of the ten audio-visual and audio modules ranged from approximately one to fifteen minutes. The full educational package is available in all formats. Data Analysis Survey data was exported from Qualtrics software and analysed using the Statistical Package for the Social Sciences (SPSS version 28). A one-way repeated measure analysis of variance was used to assess the effect of time (two weeks before, four weeks after, and 12 weeks after) on knowledge scores, with post-hoc paired t-tests used to compare time points. For non-parametric data obtained from Likert responses, the effect of time was assessed using the Friedman test with post-hoc Wilcoxon signed-rank test used to compare time points. All post-hoc tests were corrected for multiple comparisons using Bonferroni alpha corrections. To aid interpretation and discussion survey questions were thematically grouped a-priori according to each level within the NWKM - reaction, learning, and behavior (see Supplementary Table 1). The correlation between response data of thematically grouped questions were assessed using Cronbach’s alpha with α scores > 0.7 considered satisfactory [ 31 ]. Results A total of 116 participants completed Survey 1 and were provided with the educational package and an invitation to complete Survey 2 and 3. Of these participants, 52 completed Survey 2 (47% retention) and 43 completed Survey 3 (80% retention). A summary of participant characteristics is presented in Table 1 . Table 1 Descriptive summary of participant characteristics for each survey Survey number n Male Female Most common age bracket Most common years practice Most common country of graduation Principle Associate Holds a post-graduate qualification S1 116 61% 39% 35–39 years (23%) 10–14 years (31%) Australia (89%) 79% 21% 25% S2 52 59% 41% 35–39 years (22%) 10–14 years (28%) Australia (90%) 91% 9% 21% S3 43 63% 37% 35–39 years (23%) 10–14 years (26%) Australia (88%) 93% 7% 25% Level 1 - Reaction to the educational package Participant reaction to the educational package was evaluated using eight questions from Survey 3 (n = 43) grouped according to four subthemes – knowledge, behaviour, relevance, and satisfaction (Table 2 ). Most respondents viewed the educational package favorably with mean response scores ranging from 3.75 (‘I would recommend changes to my practice procedures after viewing this educational package’) and 4.43 (‘The educational package was effective at increasing my knowledge’) on a 5-point Likert scale. Most participants were either somewhat or strongly satisfied with the overall educational package (81%), and either strongly or somewhat agreed that the educational package improved their knowledge of PROMs (87%). Similarly, 72% of participants were satisfied with the duration of the package and 99% felt that the package was relevant to their needs, with 88% reporting they would recommend it to their colleagues. The correlation between subtheme questions ranged between α = 0.71 and 0.91. Table 2 Reaction to the education package Frequency of respondents (%) Survey question Mean score (1–5) Strongly disagree (1) Somewhat disagree (2) Neutral (3) Somewhat agree (4) Strongly agree (5) Knowledge The education package was effective at increasing my knowledge 4.43 0% 0% 12% 35% 53% Did the education tool improve your knowledge of PROMs? 4.32 0% 0% 14% 42% 44% Behaviour I would recommend changes to my practice procedures after viewing this education package. 3.75 5% 5% 21% 49% 21% I would recommend this education package to a colleague. 4.39 0% 0% 12% 39% 49% Relevance The education package was relevant to my needs. 4.23 0% 2% 14% 42% 42% The education package matched my learning style. 4.23 0% 2% 14% 42% 42% Satisfaction I was satisfied with the overall quality of the education package. 4.34 0% 2% 16% 28% 53% I was satisfied with the duration of the education package 4.07 0% 0% 5% 33% 39% Level 2 - Learning (Knowledge about PROMs) Participant knowledge of PROMs was evaluated in each survey using 32 questions that covered topics including PROM definitions, categories, and specific examples. The mean (SD) knowledge scores (out of 32) for Survey 1, 2, and 3 were 24.2 (6.1), 27.2 (5.5), and 27.4 (5.1), respectively. The one-way repeated measured analysis of variance (n = 43) revealed a significant effect of time on knowledge ( F 41,1 = 21.198, p < 0.001). Post-hoc t-tests indicate a significant improvement in knowledge from Survey 1 to Survey 2 (mean difference ± 95%CI = 3.0 ± 2.5, p = 0.013) and Survey 1 to Survey 3 (mean difference ± 95%CI = 3.1 ± 1.7, p < 0.001), with no significant difference in knowledge between Survey 2 and 3 (p = 1.00). Level 2 Learning – Confidence in understanding and using PROMs The related samples Friedman’s analysis of variance by ranks test (n = 39) revealed no significant effect of time on clinician confidence in: 1) understanding what a PROM is (χ 2 (2) = 0.607, p = 0.738); 2) understanding the significance of using PROMs (χ 2 (2) = 0.157, p = 0.924); 3) recognising when to apply PROMs (χ 2 (2) = 0.157, p = 0.924); 4) implementing PROMs (χ 2 (2) = 0.171, p = 0.918); or 5) knowing what patient reported measures are available (χ 2 (2) = 0.639, p = 0.726). The correlation between the questions α = 0.899. The mean ranks for each question at each time point are included in Table 3 . Table 3 Change in participant confidence using patient reported outcome measures before and after receiving the education package Survey question Survey 1 Mean rank Survey 2 Mean rank Survey 3 Mean rank How confident are you understanding what a patient reported outcome measure is? 1.95 2.06 1.99 How confident are you understanding the significance of patient reported outcome measures? 2.08 1.96 1.96 How confident are you recognising when to use patient reported outcome measures? 2.04 1.99 1.97 How confident are you implementing patient reported outcome measures? 2.01 2.03 1.96 How confident are you knowing what patient reported outcome measures are available? 1.92 2.01 2.06 Level 3 – Behaviour (Attitude towards PROMs) There was no significant effect of time on how influential PROMs were to the treatment plan and patient management (χ 2 (2) = 3.644, p = 0.162) (see Table 4 ). There was a significant effect of time on whether health professional should use PROMs using valid and reliable tools (χ 2 (2) = 6.982, p = 0.03), however, the post-hoc Wilcoxon rank-tests revealed no significance difference between Survey 1 and 2 (Z = 1.670, p = 0.095) or between Survey 1 and 3 (Z = 1.485, p = 0.138). There was no effect of time on clinician attitudes to PROMs based on the remaining statements: 1) ‘PROMs enable you to get a better understanding of your patient’s progress’ (χ 2 (2) = 1.627, p = 0.443); 2) ‘The use of validated PROMs is clinically helpful in an increasing medicolegal environment’ (χ 2 (2) = 1.853, p = 0.396); 3) ‘The use of patient reported outcome measures could be helpful in justifying ongoing treatment to third parties’ (χ 2 (2) = 4.680, p = 0.096); 4) ‘My patients are all different; therefore patient reported outcome measures would not be useful’ (χ 2 (2) = 0.628, p = 0.731); 5) ‘Available patient reported outcome measures are unsuitable for the type of patients I treat’ (χ 2 (2) = 2.086, p = 0.352); 6) ‘If I had more time, I would be interested in using patient reported outcome measures’ (χ 2 (2) = 0.609, p = 0.738); 7) ‘I do not see the use of patient reported outcome measures as a priority’ (χ 2 (2) = 2.385, p = 0.304); 8) ‘Patient reported outcome measures are unpopular with patients’ (χ 2 (2) = 1.681, p = 0.432); 9) ‘Patient satisfaction is the most important outcome’ (χ 2 (2) = 4.978, p = 0.083); 10) ‘I do not know enough about patient reported outcome measures to feel comfortable/confident using them’ (χ 2 (2) = 0.828, p = 0.661); 11) ‘The patient discontinuing treatment puts me off using patient reported outcome measures’ (χ 2 (2) = 0.023, p = 0.998); 12) ‘There is no need to change from the way that we have to assess/assessed patients’ (χ 2 (2) = 2.141, p = 0.343); 13) ‘If I had to use patient reported outcome measures, I would prefer to choose which ones I used’ (χ 2 (2) = 2.049, p = 0.359); 14) ‘Access to information about patient reported outcome measures is limited in my work environment’ (χ 2 (2) = 1.162, p = 0.559); and 15) ‘It is not necessary to measure functional outcomes’ (χ 2 (2) = 0.667, p = 0.717). The correlation between the questions α = 0.655. Table 4 Change in participant attitude towards patient reported outcome measures before and after receiving the education package Survey question Survey 1 Mean rank Survey 2 Mean rank Survey 3 Mean rank How influential are patient reported outcome measures to your treatment plan and patient management? 2.01 1.85 2.14 Health professionals should use patient reported outcome measures to monitor treatment outcomes using valid and reliable tools. 1.81 2.17* 2.03 Patient reported outcome measures enable you to get a better understanding of your patient’s progress 1.94 2.12 1.95 The use of validated patient reported outcome measures is clinically helpful in an increasing medicolegal environment. 2.12 1.92 1.96 The use of patient reported outcome measures could be helpful in justifying ongoing treatment to third parties 2.15 1.96 1.88 My patients are all different; therefore, patient reported outcome measures would not be useful 2.04 2.04 1.92 Available patient reported outcome measures are unsuitable for the type of patients I treat 2.10 2.04 1.86 If I had more time, I would be interested in using patient reported outcome measures 1.92 2.05 2.03 I do not see the use of patient reported outcome measures as a priority 2.14 1.91 1.95 Patient reported outcome measures are unpopular with patients 2.13 1.96 1.91 Patient satisfaction is the most important outcome 2.06 1.78 2.15 I do not know enough about patient reported outcome measures to feel comfortable/confident using them 2.00 1.92 2.08 The patient discontinuing treatment puts me off using patient reported outcome measures 2.00 1.99 2.01 There is no need to change from the way that we have to assess/assessed patients 1.86 2.08 2.06 If I had to use patient reported outcome measures, I would prefer to choose which ones I used 2.13 1.97 1.90 Access to information about patient reported outcome measures is limited in my work environment 2.05 2.06 1.88 Level 3 – Behaviour (Utilisation of PROMs) There was no significant effect of time on the frequency of pain-related (χ 2 (2) = 0.192, p = 0.909) or functional-related (χ 2 (2) = 3.69, p = 0.158) PROM use (Table 5 ). There was, however, a significant effect of time on the frequency of health-related PROM use (χ 2 (2) = 8.310, p = 0.016). Post-hoc Wilcoxon signed-rank tests revealed a significant increase in the mean rank score for health-related PROM use from Survey 1 (1.79) to Survey 3 (2.01) (Z = 2.707, p = 0.007, ES = 0.41) with 14 positive differences, 3 negative differences, and 27 ties (Table 3 ). There was no difference in the mean rank score between Survey 1 to Survey 2 (Z = 0.125, p = 0.901), with 10 positive differences, 7 negative differences, and 35 ties. When asked about whether the educational package increased their use of patient reported outcome measures (PROMs) in clinical practice, 39% of participants in Survey 2 answered ‘yes’ (41% No, 17% Unsure), compared to 56% in Survey 3 (28% No, 16% Unsure). Table 5 Changes in participant utilisation of patient reported outcome measures before and after receiving the education package Survey question Survey 1 Mean rank Survey 2 Mean rank Survey 3 Mean rank How often on average do you use PROMs in your everyday practice (ANY) 2.03 2.00 1.97 How often on average do you use PROMs in your everyday practice (PAIN) 2.01 1.96 2.03 How often on average do you use PROMs in your everyday practice (FUNCTIONAL) 1.86 1.95 2.19 How often on average do you use PROMs in your everyday practice (HEALTH) 1.79 2.01 2.19* *Statistically different to Survey 1 (p < 0.05) Discussion This longitudinal study evaluated the impact of an educational package developed and implemented using the KTA framework, and which aimed to increase PROM knowledge and utilisation in a small cohort of chiropractors in Australia. The results are presented and discussed according to three of the four levels of the NWKM – reaction, learning, and behavior [32]. Overall, the reaction to the educational package was very positive, with most participants noting improved knowledge and a high level of satisfaction with the material. Although there was a significant improvement in knowledge after receiving the educational package, which was retained at 12 weeks, these changes were small and did not translate to improvements in confidence, attitude, or frequency of PROM usage. The findings suggest that changes in knowledge alone may be insufficient to influence the frequency of PROM utilisation by chiropractors. Level 1 - Reaction It is important to evaluate the participant’s reactions to the course or training when evaluating the success of the educational intervention [33]. Overall, the findings confirmed a positive reaction to the educational package. Most respondents reported that the intervention was favourable, engaging, and relevant to their needs. These findings could therefore be important to create clinical behaviour change as Oreg, Vakola [34] found that organizational and behaviour change are not well implemented unless there is a positive reaction. Survey 1 included items which allowed respondents to communicate directly with the content creators and inform design and content of the educational package [28, 35]. Importantly, 98% of those surveyed felt that the educational package they engaged with was highly relevant to their needs and matched their learning style; the latter of which has been associated with improved learning outcomes in educational [36]. Furthermore, 81% of the survey respondents were either somewhat or strongly satisfied with the overall educational package and 88% stated that the educational package was effective at increasing their knowledge and they would recommend it to their colleagues. Interestingly, only 70% of the respondents noted that they would recommend changes to their practice procedures after viewing the educational package. While research suggests that participant reaction to learning is important such measures may not correlate to how much participants learnt, or whether their behaviour changed [37-40]. For example, a recent study of acupuncturists noted that while the reaction to the training is important, levels 2 (learning) and 3 (behaviour) of the NWKM are the most relevant to determine the success of the program [41]. Level 2 - Learning Level two of the NWKM is concerned with determining the degree of learning, knowledge, confidence, and skill acquisition because of training [26, 32, 42, 43]. To assess knowledge, evaluation of the participant’s recall, understanding and application of the learning should be measured [44]. Recall and understanding were assessed using three questions included in each survey and covered a range of topics including the definition of PROMs, categories of PROMs, and the identification of PROMs examples using multiple-choice questions. The results show a significant improvement in knowledge scores between survey 1 and survey 2, which were retained at survey 3, suggesting the intended learning outcomes were achieved and the educational package was successful [45, 46]. Nevertheless, the improvements in knowledge were modest (~10%), and while statistically significant, may be insufficient to affect a change in participant behavior and attitude towards PROMs. Despite a recent survey of chiropractors in Australia suggesting that improving clinician understanding of PROMs and why/when to use PROMs would improve utilisation rates, our study, together with others, suggest that knowledge, in and of itself, may not be sufficient to change behavior [47, 48]. Similarly, confidence of the learner, which has been suggested to be a gap between learning and behaviour [49], was also not changed as a result of the educational package. We found no difference in confidence scores between survey 1, and surveys 2 and 3. Research suggests that motivation and confidence are key determinants to create behavioral change [50]. Boyce, Robertson [51] further suggest that building confidence is important if intentions are to be translated into behaviour change. Nevertheless, while confidence may bridge the gap between knowledge and behavior, improving knowledge alone seems insufficient to change confidence. Overall, this study suggests that the provision of an online educational package aimed to improve knowledge of PROMs in chiropractors had no effect on their confidence in understanding what PROMs were available or when and/or how to use them. Level 3 - Behaviour Level 3 of the NWKM evaluates to what extent the newly acquired knowledge or skills have been practically applied [42, 52]. This level also assesses the degree to which the participants apply what they learned and modified their behaviour based on the intervention [53]. In the current study we evaluated behavior change using questions related to two subthemes: 1) attitude toward PROMs; and 2) frequency of PROM utilisation. Participant’s attitude toward PROMs were evaluated using 16 questions, which covered a variety of attitudinal aspects relating to the importance and influence of PROMs in clinical practice. Of the 16 questions, only one question was significantly different after the provision of the educational package. Participants more strongly agreed with the statement: ‘Health professionals should use patient reported outcome measures to monitor treatment outcomes using valid and reliable tools’ in survey 2, compared to survey 1. The educational package may have highlighted the arrival of the “era of accountability” with more pressure placed on health care providers to provide treatment evidence [54]. The last decade has seen an increasing focus and interest not only in a patients symptoms, but also in documenting the patient experience and their interactions with healthcare providers [55]. Nevertheless, the overall results do not support a change in clinician attitudes toward PROMs after receiving the educational package. Whilst difficult to determine, it is likely that respondents already had a very favorable view towards PROMs and so there may have been a ceiling effect limiting the impact of the educational package on participant attitudes. Previous studies do show that most chiropractors have a favorable view of PROMs [5, 15], however, these views do not necessarily correlate to higher levels of usage [15]. Although 70% of the respondents reported that they would recommend changes to their practice procedures after viewing the educational package, the data did not support that those changes occurred. Furthermore, despite most respondents acknowledging the importance of PROMs and the need for clinical change to include PROMs, the clinical implementation did not increase after receiving the educational package. Except for health-related PROMs, the reported frequency of PROM utilisation did not change from survey 1 to surveys 2 and 3. We found a small but significant increase in the reported use of health-related PROMs from survey 1 to survey 3. Although there was a similar trend toward increased use of functional PROMs, these differences were not significant. These improvements may be explained, at least in part, by two factors. First, the current study, as well as previous studies [8], show that functional and health-related PROMs are less frequently used by chiropractors in clinical practice, compared to pain-related PROMs, and so their frequency of use may be more likely changed with an educational package that promotes awareness of these PROMs. Secondly, and further to this point, the educational package was specifically designed to address these practice gaps by providing a comprehensive summary of numerous health- and functional-related PROMs relevant to each body region. It is possible that much of this content may have been perceived as ‘new and novel’ and therefore may be more likely to elicit a measurable change in behaviour, compared to the more commonly used pain-related PROMs. Overall, the educational package did not affect a measurable change in participant’s confidence, attitude, or frequency of PROM utilisation. Although we used the KTA framework to design the content of the educational package, we did not include any specific strategies to promote translation, but rather relied on knowledge to drive behavior change. Clearly, knowledge alone is not sufficient to affect behavioral change. Although knowledge has been consistently identified as a barrier/facilitator to PROM utilisation in healthcare, multiple other barriers exist which were not addressed in the current study and may be more influential in changing behavior. Time was the most reported barrier to PROM usage by chiropractors [5,8]. Although the educational package provided links to source material there were no specific implementation strategies included in the package that may have eased the time burden of using PROMs in clinical practice. The provision of all relevant PROMs to chiropractic care, including background information, via a single electronic application may be a future strategy to better align PROM knowledge with PROM access, and reduce the time the burden of accessing PROMs in a clinical setting. It is possible that if this study had continued over a longer period of time, this may have increased the likelihood of measuring behavioral change. Axtell, Maitlis [56] found that the amount of learning transferred into practice one month post intervention was a strong predictor for learning transferred one year post intervention. According to Kirkpatrick D [57] the most accurate time to evaluate behaviour change is at least three months after the training was applied, although literature suggests that behaviour change may not occur until six months, to enable learners sufficient time to put their new skills into practice [58]. Therefore, a longer-term follow-up (up to 12 months) may have been needed to identify whether the short-term changes in knowledge changed behaviour. Limitations In addition to the short follow-up period, there are several other limitations to this study. First, the low response rate to the cohort survey may limit the strength and generalisability of the findings. Although online surveys are a common research method, their success, particularly amongst health care professionals has been questioned [59]. Cunningham, Quan [59] imply that low survey response rates are common within the health care profession. Another limitation is bias, which is an inherent issue in the design of surveys [60]. Although the authors aimed to reduce selection bias and population bias by inviting all members of Australia’s main chiropractic associations to participate, an unintentional responder bias may have occurred. Due to the low response rate, the survey respondents may not be representative of the entire chiropractic profession in Australia. A further limitation of this study was that there was no control group. Additionally, given the substantial drop-out between survey 1 and survey 2, it is possible the participants who completed all three surveys may have already had favorable views of PROMs, and relatively high utilisation rates, and so there may have been a potential ceiling effect associated with the educational package that affected changes in confidence, attitude, and frequency of PROM usage. Conclusion An online educational package delivered to chiropractors in Australia was effective at improving knowledge of PROMs, including what PROMs are and why, how, and when they should be used in clinical practice. Despite modest improvements in knowledge, which were retained 3-months after the educational package was provided, there was no evidence that participants confidence, attitudes, or frequency of PROM use changed because of the intervention. This study suggests that knowledge of PROMs alone may be insufficient to change the frequency of their use in clinical practice by chiropractors in Australia. Declarations Ethical Approval and Consent to participate Full ethical approval was granted by the Human Research Ethics Committee of Central Queensland University, reference number 0000022391. Consent for publication Not applicable Availability of data and materials Not applicable Competing interests The authors declare they have no competing interests Funding Not applicable Authors' contributions Concept development: NCC, SO, AGS. Design (planned the methods to generate the results): NCC, SO, AGS, GB. Ethics submission/approval NCC, AGS. Analysis and interpretation of data: SO, NCC, AGS, GB. Manuscript preparation NCC, SO, AGS, GB. Critical review: SO, AGS, GB. All authors read and approved the final manuscript. Acknowledgements Not applicable Authors' information(optional) 1 College of Health Sciences, School of Health, Medical and Applied Sciences, Central Queensland University, Branyan, QLD, Australia. 2 School of Applied Psychology, Griffith University, South Brisbane, QLD, Australia. References Santana MJ, Haverman L, Absolom K, Takeuchi E, Feeny D, Grootenhuis M, et al. Training clinicians in how to use patient-reported outcome measures in routine clinical practice. Qual Life Res. 2015;24(7):1707-18. doi: 10.1007/s11136-014-0903-5 Thompson L, Bekelman D, Allen L, Peterson P. Patient-Reported Outcomes in Heart Failure: Existing Measures and Future Uses. Current Heart Failure Reports. 2015;12(3):236-46. doi:10.1007/s11897-015-0253-9 Wolpert M. Uses and Abuses of Patient Reported Outcome Measures (PROMs): Potential Iatrogenic Impact of PROMs Implementation and How It Can Be Mitigated. Administration and Policy in Mental Health and Mental Health Services Research. 2014;41(2):141-5. doi: 10.1007/s10488-013-0509-1 Clohesy N, Schneiders A, Eaton S. Utilization of Low Back Pain Patient Reported Outcome Measures Within Chiropractic Literature: A Descriptive Review. Journal of Manipulative and Physiological Therapeutics. 2018;41(7):628-39. doi: https://doi.org/10.1016/j.jmpt.2017.11.007 Holmes M, Bishop F, Newell D, Field J, Lewith G. Chiropractors' views on the use of patient-reported outcome measures in clinical practice: a qualitative study. Chiropr Man Therap. 2018;26:50. doi: https://doi.org/10.1186/s12998-018-0219-6 Weldring T, Smith SMS. Patient-Reported Outcomes (PROs) and Patient-Reported Outcome Measures (PROMs). Health Serv Insights. 2013;6:61-8. doi: 10.4137/HSI.S11093 Yeomans S, Liebenseon C. Applying outcome measures. JNMS. 1997;5. Clohesy N, Schneiders A. A preliminary investigation examining patient reported outcome measures for low back pain and utilisation amongst chiropractors in Australia: facilitators and barriers to clinical implementation. Chiropr Man Therap. 2018;26:38. doi: https://doi.org/10.1186/s12998-018-0208-9 Brinkman M, Barten D, Pisters M, Verheij R. Current use of PROMs and factors associated with their use in patients with nonspecific low back pain. Learning Health Systems. 2019;3(4):e10194. doi:10.1002/lrh2.10194 Duncan E, Murray J. The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review. BMC Health Services Research. 2012;12(1):96. doi: https://doi.org/10.1186/1472-6963-12-96 Greenhalgh J, Meadows K. The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. J Eval Clin Pract. 1999;5(4):401-16. doi: 10.1046/j.1365-2753.1999.00209.x. Antunes B, Harding R, Higginson I. Implementing patient-reported outcome measures in palliative care clinical practice: a systematic review of facilitators and barriers. Palliat Med. 2014;28(2):158-75. doi: 10.1177/0269216313491619. Chapman-Smith D. The Role of Patient Questionnaires. The Chiropractic Report. 2012;26(5). Eilayyan O, Visca R, Zidarov D, Ware P, Bussières A, Ahmed S. Developing theory-informed knowledge translation strategies to facilitate the use of patient-reported outcome measures in interdisciplinary low back pain clinical practices in Quebec: mixed methods study. BMC Health Services Research. 2020;20(1):789. doi: https://doi.org/10.1186/s12913-020-05616-5 Clohesy N, Schneiders A, Barbery G, Obst S. Clinicians’ perceived value and demographic factors that predict the utilisation of patient reported outcome measures for low back pain amongst chiropractors in Australia. Chiropr Man Therap. 2021;29(1):42. doi: https://doi.org/10.1186/s12998-021-00399-w Nilsson E, Orwelius L, Kristenson M. Patient-reported outcomes in the Swedish National Quality Registers. J Intern Med. 2016;279(2):141-53. doi: 10.1111/joim.12409 Lynch E, Mudge A, Knowles S, Kitson A, Hunter S, Harvey G. "There is nothing so practical as a good theory": a pragmatic guide for selecting theoretical approaches for implementation projects. BMC Health Serv Res. 2018;18(1):857. doi: https://doi.org/10.1186/s12913-018-3671-z Field B, Booth A, Ilott I, Gerrish K. Using the Knowledge to Action Framework in practice: a citation analysis and systematic review. Implementation Science. 2014;9(1):172. doi: https://doi.org/10.1186/s13012-014-0172-2 Davison C, Ndumbe-Eyoh C, C. Critical examination of knowledge to action models and implications for promoting health equity. International journal for equity in health. 2015;14(49). doi: https://doi.org/10.1186/s12939-015-0178-7 Moore J, Virva R, Henderson C, Lenca L, Butzer J, Lovell L, et al. Applying the Knowledge-to-Action Framework to Implement Gait and Balance Assessments in Inpatient Stroke Rehabilitation. Archives of Physical Medicine and Rehabilitation. 2020;103(7S):S230-54. doi: 10.1016/j.apmr.2020.10.133. Graham I, Tetroe J. Some theoretical underpinnings of knowledge translation. Acad Emerg Med. 2007;14(11):936-41. doi: 10.1197/j.aem.2007.07.004. Graham I, Logan J, Harrison M, Straus S, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13-24. doi: 10.1002/chp.47. Sinden K, MacDermid J. Does the knowledge-to-action (KTA) framework facilitate physical demands analysis development for firefighter injury management and return-to-work planning? J Occup Rehabil. 2014;24(1):146-59. doi: 10.1007/s10926-013-9442-0. Kirkpatrick W, Kirkpatrick J. Four levels of training evaluation. 2016. United States, ATD Press. Gill S, Sharma G. Evaluation of Vocational Training Program from the Trainees' Perspective: An Empirical Study. Pacific Business Review International. 2013;6(5):35-43. Rouse D. Employing Kirkpatrick's evaluation framework to determine the effectiveness of health information management courses and programs. Perspect Health Inf Manag. 2011;8. AHPRA. Chiropractic board statistics 2020. https://www.chiropracticboard.gov.au/About-the-Board/Annual-report.aspx. Accessed 25 March 2022. Bradwell P, Marr S. Making the most of collaboration: an international survey of public service co-design, Demos Report 23. London: 2008. O'Cathain A, Thomas KJ. "Any other comments?" Open questions on questionnaires – a bane or a bonus to research? BMC Medical Research Methodology. 2004;4(1):25. doi: https://doi.org/10.1186/1471-2288-4-25 Smith A, Ling P, Hill D. The Adoption of Multiple Modes of Delivery in Australian Universities. Journal of University Teaching & Learning Practice. 2006;3(2). doi: https://doi.org/10.53761/1.3.2.2 Bland JM, Altman DG. Statistics notes: Cronbach's alpha. BMJ. 1997;314(7080):572. doi: https://doi.org/10.1136/bmj.314.7080.572 Cahapay M. International Journal of Assessment Tools in Education. Int J Assess Tools Educ. 2021;8(1):135–44. doi:https://doi.org/10.21449/ijate.856143 Gökhan Ö. Program Evaluation through Kirkpatrick's Framework. Pacific Business Review International. 2015;8(1). Oreg S, Vakola M, Armenakis A. Change Recipients’ Reactions to Organizational Change: A 60-Year Review of Quantitative Studies. The Journal of Applied Behavioral Science. 2011;47(4):461-524. doi: https://doi.org/10.1177/0021886310396550 Shahvari Z, Raisi F, Firoozi Khojastefar R, Yahyavi S, Mirsepassi Z. Memos on Sexuality Education Effectiveness; A Qualitative Study in Severe Mental Illness in Iran. International Journal of Sexual Health. 2020;32(3):254-66. doi: https://doi.org/10.1080/19317611.2020.1793047 Baykan Z, Naçar M. Learning styles of first-year medical students attending Erciyes University in Kayseri, Turkey. Advances in Physiology Education. 2007;31(2):158-60. doi: 10.1152/advan.00043.2006 Alliger G, Janak E. Kirkpatrick's levels of training criteria: Thirty years later. Personnel Psychology. 1989;42(2):331-42. doi: doi/abs/10.1111/j.1744-6570.1989.tb00661.x Arthur W, Bennett W, Edens P, Bell S. Effectiveness of Training in Organizations. Journal of Applied Psychology. 2003;88(2):234-45. doi: https://doi.org/10.1037/0021-9010.88.2.234 Colquitt J, LePine J, Noe R. Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology. 2000;85(5):678–707. doi: 10.1037//0021-9010.85.5.678 Tannenbaum SI, Yukl G. Training and development in work organizations. Annual Review of Psychology. 1992;43:399-441. doi: https://doi.org/10.1146/annurev.ps.43.020192.002151 Liao S-C, Hsu S-Y. Evaluating A Continuing Medical Education Program: New World Kirkpatrick Model Approach. International Journal of Management, Economics and Social Sciences. 2019;8. doi:10.32327/IJMESS.8.4.2019.17 Phillips J, Phillips P. Handbook of Training Evaluation and Measurement Methods (4th). London: 2016. doi: https://doi.org/10.4324/9781315757230 Allen LM, Hay M, Palermo C. Evaluation in health professions education—Is measuring outcomes enough? Medical Education. 2022;56(1):127-36. doi: 10.1111/medu.14654 Kirkpatrick J, Kirkpatrick W. Introduction to the New World Kirkpatrick Model. 2019. https://www.kirkpatrickpartners.com/wp-content/uploads/2021/11/Introduction-to-the-Kirkpatrick-New-World-Model.pdf. Accessed on 6 January 2022. Deodhar M, Powdwal S. Impact of continuing education programs (CEPs) on LIS professionals in academic libraries in Mumbai, India. Library management. 2017;38(2/3):117-30. Mohamed R, Ariyani A, Alias S. Evaluating the Effectiveness of a Training Program Using the Four Level Kirkpatrick Model in the Banking Sector in Malaysia: In: 3rd International Conference on Business and Economic Research, 12-13 March. Bandung, Indonesia2012. Arlinghaus KR, Johnston CA. Advocating for Behavior Change With Education. Am J Lifestyle Med. 2017;12(2):113-6. doi:10.1177/1559827617745479 Worsley A. Nutrition knowledge and food consumption: can nutrition knowledge change food behaviour? Asia Pacific journal of clinical nutrition. 2002(11):579-85. doi: 10.1046/j.1440-6047.11.supp3.7.x. Piryani RM, Dhungana GP, Piryani S, Sharma Neupane M. Evaluation of teachers training workshop at Kirkpatrick level 1 using retro-pre questionnaire. Adv Med Educ Pract. 2018;9:453-457. doi:10.2147/AMEP.S154166. Lubman D, Hall K, Gibbie T. Motivational interviewing techniques. Australian Journal for General Practitioners. 2012;41:660-7. Boyce T, Robertson R, Dixon A. Commissioning and behaviour Change Kicking Bad Habits final report. 2008. Baldwin T, Ford J. Transfer of Training: A Review and Directions of Future Research. Personnel Psychology. 1988;41(1):63–105. doi: https://doi.org/10.1111/j.1744-6570.1988.tb00632.x Cheng E, Hampson I. Transfer of training: A review and new insights Journal of Management Reviews 2008;10(4):327–41. doi: https://doi.org/10.1111/j.1468-2370.2007.00230.x Relman A. Assessment and accountability : the third revolution in medical care. 2012. 87-90. doi: 10.1056/NEJM198811033191810. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1):e001570. doi:10.1136/bmjopen-2012-001570 Axtell CM, Maitlis S, Yearta SK. Predicting immediate and longer-term transfer of training. Personnel review. 1997;26(3):201-13. doi: http://dx.doi.org/10.1108/00483489710161413 Kirkpatrick D. Great Ideas Revisited. Techniques for Evaluating Training Programs. Revisiting Kirkpatrick's Four-Level Model. Training and Development. 1996;50(1). May G, Kahnweiler W. The effect of a mastery practiice design on learning and transfer in behavior modelling training Personnel psychology. 2000;53(2):353-73. doi: https://doi.org/10.1111/j.1744-6570.2000.tb00205.x Cunningham CT, Quan H, Hemmelgarn B, Noseworthy T, Beck CA, Dixon E, et al. Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol. 2015;15. doi:10.1186/s12874-015-0016-z Choi B, Pak A. A catalog of biases in questionnaires. Prev Chronic Dis. 2005;2. Supplementary Files Supplementarytable.docx Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Minor Revision 28 Aug, 2022 Reviewers agreed at journal 16 Aug, 2022 Reviewers invited by journal 03 Aug, 2022 Editor invited by journal 02 Aug, 2022 Editor assigned by journal 02 Aug, 2022 First submitted to journal 28 Jul, 2022 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1907999","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":126029918,"identity":"27336c2a-e50f-4944-bc16-70f13c9444da","order_by":0,"name":"Natalie Clohesy","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABFUlEQVRIiWNgGAWjYLCCBwZAgp2BDcSWgwgZWOBRz8zAkGAApsFajKFaJAhoYUBoSWyACOPWYs5+/uCHhAIbBv5m5mMPPvw5nN7fnmO64UeBBAN/e3cCNi2WPcnMEgkGaQwSh9nSDWe2Hc6dceaN2c0eoMMkzpzdgE2LwYFkBqCWwwwGzDxm0rwNh3M3SOSY3eABajGQyMWu5fxj5h8JBv8hWv4AHWYA1HLzDz4tN5LZgLYcgGhhYDucANJyG68tNx6bWSQYJPMA/ZIm2duWbjjjzLOy2zIGEjw4/XI+8fGND3/s5Pjbm49J/PhjLc/fnrzt5ps/NkCRXqxaYIAHSjczQKMJLkIQ1MG1jIJRMApGwSiAAQBRSFvFOW9iZQAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0002-2330-5101","institution":"Central Queensland University","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Natalie","middleName":"","lastName":"Clohesy","suffix":""},{"id":126029919,"identity":"444bfdfb-32df-4dc1-99bf-6f2818d85ddf","order_by":1,"name":"Anthony Schneiders","email":"","orcid":"","institution":"","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Anthony","middleName":"","lastName":"Schneiders","suffix":""},{"id":126029920,"identity":"d3ceea96-3af0-40e1-9c70-31242138b144","order_by":2,"name":"Gaery Barbery","email":"","orcid":"","institution":"","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Gaery","middleName":"","lastName":"Barbery","suffix":""},{"id":126029921,"identity":"948c4b11-f39a-4950-9e6d-68a7ea2d6162","order_by":3,"name":"Steven Obst","email":"","orcid":"","institution":"","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Steven","middleName":"","lastName":"Obst","suffix":""}],"badges":[],"createdAt":"2022-07-29 07:49:19","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1907999/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1907999/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":24799523,"identity":"a3cb280e-a578-467a-86d3-614e3bc7026b","added_by":"auto","created_at":"2022-08-04 18:06:25","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":445555,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1907999/v1/f9f071ae-ee70-4393-a5dd-3c4de3ac385e.pdf"},{"id":24799522,"identity":"565bcc53-1e7b-448b-8c8a-bcc8d0b290bb","added_by":"auto","created_at":"2022-08-04 18:06:22","extension":"docx","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":18543,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementarytable.docx","url":"https://assets-eu.researchsquare.com/files/rs-1907999/v1/e9fc3acadaa339b85ecc6fc0.docx"}],"financialInterests":"","formattedTitle":"The impact of a targeted education package on the knowledge, attitudes, and utilisation of patient reported outcome measures amongst chiropractors in Australia.","fulltext":[{"header":"Background","content":"\u003cp\u003ePatient reported outcome measures (PROMs) are validated questionnaires and survey instruments used to measure the status of a patient\u0026rsquo;s health condition using information that comes directly from the patient [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. PROMs were initially used as epidemiological surveys to identify patterns of symptomatology or health status, however, they have evolved to be tools that can also be used to support clinical practice and research [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. In the clinical setting, PROMs can assist the decision-making process, improve therapist-patient communication, monitor treatment progress, and/or facilitate patient-centered care [\u003cspan additionalcitationids=\"CR5 CR6\" citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. PROMs are therefore considered an integral part of evidence-based health care and therefore it is important that clinicians are equipped with the requisite skills and knowledge needed to apply PROMs in clinical practice.\u003c/p\u003e \u003cp\u003eA recent survey of chiropractors in Australia found that 72% of respondents reported using PROMs for the management of low back pain (LBP) [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. Similar rates of PROM utilisation were reported for physiotherapists in the United Kingdom, where 72% of physiotherapists, and 71% of physiotherapy practices, routinely used PROMs [\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]. Despite these relatively high rates, it has been suggested there is a lack of knowledge and understanding of PROMs amongst chiropractors, which may limit the full clinical implementation of PROMs and their potential benefits [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan additionalcitationids=\"CR9 CR10\" citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]. For example, a survey of chiropractors in Australia found that 19% of respondents reported that a lack of knowledge regarding PROMs prevented clinical implementation, while time required to administer and score PROMs in a clinical setting (17%) was also a barrier to their utilisation [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. The results of studies by Holmes, Bishop [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e] and Antunes, Harding [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e] mirrored these findings, suggesting that a clinician\u0026rsquo;s lack of PROM knowledge was a significant barrier to implementation.\u003c/p\u003e \u003cp\u003eAlthough chiropractors are increasingly encouraged to use PROMs [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e], a lack of knowledge transfer from literature, education, and training into clinical practice are likely barriers to their successful implementation into clinical practice [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]. Furthermore, even though chiropractors place high value on the importance of PROMs in clinical practice, these beliefs do not appear to translate to increased frequency of PROM usage, suggesting other factors, including knowledge, may be central to changing behaviors [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eTo improve the transfer of knowledge to practice, a number of theories, models and frameworks have been developed [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. Within allied health settings, the most highly cited and accepted model is the Knowledge to Action framework (KTA) [\u003cspan additionalcitationids=\"CR18 CR19\" citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e] created by Graham and Tetroe [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]. KTA is a conceptual framework made up of two distinct components - knowledge creation and the action cycle. The knowledge creation phase is represented as a tunnel and the information becomes more refined and specific as it passes through this phase to uncover the most valid and useful knowledge [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]. The action phase is a cycle leading to implementation or application of knowledge uncovered in the knowledge creation phase. During this step, adapting knowledge to meet the needs within particular contexts or populations occurs [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe aim of this study was to evaluate the impact of an educational package developed and implemented using KTA framework to improve PROM knowledge and utilisation in a cohort of chiropractors in Australia. We hypothesised that a tailored educational package intervention would increase participant knowledge of PROMs, which would translate to a positive change in attitude and frequency of PROM utilisation in clinical practice.\u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003eStudy Design\u003c/h2\u003e \u003cp\u003eA longitudinal interventional cohort survey with repeated measures was used to gain an understanding of the respondent\u0026rsquo;s knowledge, attitudes, and utilisation of PROMs in the Chiropractic profession of Australia. An online educational package was designed, delivered and the findings evaluated using the New World Kirkpatrick Model (NWKM) [\u003cspan additionalcitationids=\"CR25\" citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e] using three of the four key themes of the model: reaction, learning, and behaviour. The study was conducted over 7 months and included three online surveys administered at specific timepoints using the Qualtrics software. Full ethical approval was granted by the Human Research Ethics Committee of Central Queensland University, reference number 0000022391.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003eParticipants\u003c/h2\u003e \u003cp\u003eChiropractors who were current members of the Australian Chiropractic Association (ACA) or Chiropractic Australia (CA) were invited to participate in the study via member emails and advertisement in the association\u0026rsquo;s newsletters. The sample population consisted of 3,215 ACA members and 1,163 CA members, totaling 4,378 potential participants, representing the majority of Chiropractors registered and practicing in Australia in 2020 [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e]. The recruitment phase and data collection period occurred from November 2020 \u0026ndash; May 2021.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003eLongitudinal Surveys\u003c/h2\u003e \u003cp\u003eParticipants were surveyed on three occasions: 1) two weeks before the educational package (Survey 1); 2) four weeks after receiving the educational package (Survey 2); and 3) 12 weeks after receiving the educational package (Survey 3). Survey 1 collected demographic information including age, sex, country of graduation, highest level of education, in addition to the participant\u0026rsquo;s knowledge and attitudes about PROMs and their preferred mode of delivery (e.g., audio, audio-visual or transcript) for the educational package. Survey 2 and 3 repeated the knowledge and attitude questions from Survey 1, while omitting the demographic questions, and including items assessing the participant\u0026rsquo;s reaction to the educational package.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eEducational Package\u003c/h2\u003e \u003cp\u003eThe content and design of the educational package was informed by data obtained from Survey 1, and therefore, being co-designed [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. Respondents were given multiple-choice questions with options of content to be included in the educational package. There was also an open text box to allow them to formulate their own answers and request specific information to be included, which is recommended when using survey methods [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e]. Survey 1 also asked the respondents about their preference of mode for the delivery of the educational package. Of the 116 respondents, 81% were very likely/likely to view via text (electronic), 57% via audio-video format, 55% via text (paper), and 35% via audio. Given the spread of preferred formats the educational package was adapted and made available via for all three modes \u0026ndash; audio-visual, audio and text [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eWhen asked about what topics should be included in the package, most respondents requested \u0026lsquo;Examples of the best PROMs for major conditions\u0026rsquo; (85%), \u0026lsquo;How PROMs can be used in Chiropractic clinical practice (83%)\u0026rsquo;, and \u0026lsquo;Explanation of categories of PROMs (80%)\u0026rsquo;. The subsequent educational package consisted of 10 modules: 1) introduction; 2) background of PROMs; 3) PROMs in Chiropractic practice; 4) non-location specific PROMs; 5) PROMs for headache, whiplash, temporomandibular joint and dizziness; 6) PROMs for cervical spine; 7) PROMs for thoracic spine; 8) PROMs for lumbar spine; 9) PROMs for upper extremity; and 10) PROMs for lower extremity. For modules 4\u0026ndash;10, the following topics were included (where available and/or appropriate) for each example PROM: category, overview, scoring, area of assessment, method of delivery, and clinical condition(s). The duration of the ten audio-visual and audio modules ranged from approximately one to fifteen minutes. The full educational package is available in all formats.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003eData Analysis\u003c/h2\u003e \u003cp\u003eSurvey data was exported from Qualtrics software and analysed using the Statistical Package for the Social Sciences (SPSS version 28). A one-way repeated measure analysis of variance was used to assess the effect of time (two weeks before, four weeks after, and 12 weeks after) on knowledge scores, with post-hoc paired t-tests used to compare time points. For non-parametric data obtained from Likert responses, the effect of time was assessed using the Friedman test with post-hoc Wilcoxon signed-rank test used to compare time points. All post-hoc tests were corrected for multiple comparisons using Bonferroni alpha corrections. To aid interpretation and discussion survey questions were thematically grouped a-priori according to each level within the NWKM - reaction, learning, and behavior (see Supplementary Table\u0026nbsp;1). The correlation between response data of thematically grouped questions were assessed using Cronbach\u0026rsquo;s alpha with α scores\u0026thinsp;\u0026gt;\u0026thinsp;0.7 considered satisfactory [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003eA total of 116 participants completed Survey 1 and were provided with the educational package and an invitation to complete Survey 2 and 3. Of these participants, 52 completed Survey 2 (47% retention) and 43 completed Survey 3 (80% retention). A summary of participant characteristics is presented in Table \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\u0026nbsp;\u003ctable border=\"1\" id=\"Tab1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eDescriptive summary of participant characteristics for each survey\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey number\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003en\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eMale\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eFemale\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eMost common age bracket\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eMost common years practice\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eMost common country of graduation\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003ePrinciple\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAssociate\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eHolds a post-graduate qualification\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eS1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e116\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e61%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e39%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e35\u0026ndash;39 years\u003c/p\u003e\n \u003cp\u003e(23%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10\u0026ndash;14 years\u003c/p\u003e\n \u003cp\u003e(31%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAustralia\u003c/p\u003e\n \u003cp\u003e(89%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e79%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e21%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e25%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eS2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e52\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e59%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e41%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e35\u0026ndash;39 years\u003c/p\u003e\n \u003cp\u003e(22%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10\u0026ndash;14 years\u003c/p\u003e\n \u003cp\u003e(28%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAustralia\u003c/p\u003e\n \u003cp\u003e(90%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e91%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e9%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e21%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eS3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e43\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e63%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e37%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e35\u0026ndash;39 years\u003c/p\u003e\n \u003cp\u003e(23%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10\u0026ndash;14 years\u003c/p\u003e\n \u003cp\u003e(26%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAustralia\u003c/p\u003e\n \u003cp\u003e(88%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e93%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e7%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e25%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u003c/p\u003e\n\u003cdiv class=\"Section2\" id=\"Sec9\"\u003e\n \u003ch2\u003eLevel 1 - Reaction to the educational package\u003c/h2\u003e\n \u003cp\u003eParticipant reaction to the educational package was evaluated using eight questions from Survey 3 (n\u0026thinsp;=\u0026thinsp;43) grouped according to four subthemes \u0026ndash; knowledge, behaviour, relevance, and satisfaction (Table \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e). Most respondents viewed the educational package favorably with mean response scores ranging from 3.75 (\u0026lsquo;I would recommend changes to my practice procedures after viewing this educational package\u0026rsquo;) and 4.43 (\u0026lsquo;The educational package was effective at increasing my knowledge\u0026rsquo;) on a 5-point Likert scale. Most participants were either somewhat or strongly satisfied with the overall educational package (81%), and either strongly or somewhat agreed that the educational package improved their knowledge of PROMs (87%). Similarly, 72% of participants were satisfied with the duration of the package and 99% felt that the package was relevant to their needs, with 88% reporting they would recommend it to their colleagues. The correlation between subtheme questions ranged between \u0026alpha;\u0026thinsp;=\u0026thinsp;0.71 and 0.91.\u003c/p\u003e\u0026nbsp;\u003ctable border=\"1\" id=\"Tab2\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eReaction to the education package\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\u0026nbsp;\u003c/th\u003e\n \u003cth align=\"left\"\u003e\u0026nbsp;\u003c/th\u003e\n \u003cth align=\"left\" colspan=\"5\"\u003e\n \u003cp\u003eFrequency of respondents (%)\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSurvey question\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMean score\u003c/p\u003e\n \u003cp\u003e(1\u0026ndash;5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStrongly disagree\u003c/p\u003e\n \u003cp\u003e(1)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSomewhat disagree\u003c/p\u003e\n \u003cp\u003e(2)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNeutral\u003c/p\u003e\n \u003cp\u003e(3)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSomewhat agree\u003c/p\u003e\n \u003cp\u003e(4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStrongly agree\u003c/p\u003e\n \u003cp\u003e(5)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"Underline\" name=\"Emphasis\" type=\"Underline\"\u003eKnowledge\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eThe education package was effective at increasing my knowledge\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.43\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e12%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e35%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e53%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDid the education tool improve your knowledge of PROMs?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.32\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e14%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e42%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e44%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"Underline\" name=\"Emphasis\" type=\"Underline\"\u003eBehaviour\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eI would recommend changes to my practice procedures after viewing this education package.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3.75\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e21%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e49%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e21%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eI would recommend this education package to a colleague.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.39\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e12%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e39%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e49%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"Underline\" name=\"Emphasis\" type=\"Underline\"\u003eRelevance\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eThe education package was relevant to my needs.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.23\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e14%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e42%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e42%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eThe education package matched my learning style.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.23\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e14%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e42%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e42%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e\u003cspan class=\"Underline\" name=\"Emphasis\" type=\"Underline\"\u003eSatisfaction\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003ctd align=\"left\"\u003e\u0026nbsp;\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eI was satisfied with the overall quality of the education package.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.34\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e16%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e28%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e53%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eI was satisfied with the duration of the education package\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e4.07\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e5%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e33%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e39%\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec10\"\u003e\n \u003ch2\u003eLevel 2 - Learning (Knowledge about PROMs)\u003c/h2\u003e\n \u003cp\u003eParticipant knowledge of PROMs was evaluated in each survey using 32 questions that covered topics including PROM definitions, categories, and specific examples. The mean (SD) knowledge scores (out of 32) for Survey 1, 2, and 3 were 24.2 (6.1), 27.2 (5.5), and 27.4 (5.1), respectively. The one-way repeated measured analysis of variance (n\u0026thinsp;=\u0026thinsp;43) revealed a significant effect of time on knowledge (\u003cem\u003eF\u003c/em\u003e\u003csub\u003e41,1\u003c/sub\u003e = 21.198, p\u0026thinsp;\u0026lt;\u0026thinsp;0.001). Post-hoc t-tests indicate a significant improvement in knowledge from Survey 1 to Survey 2 (mean difference\u0026thinsp;\u0026plusmn;\u0026thinsp;95%CI\u0026thinsp;=\u0026thinsp;3.0\u0026thinsp;\u0026plusmn;\u0026thinsp;2.5, p\u0026thinsp;=\u0026thinsp;0.013) and Survey 1 to Survey 3 (mean difference\u0026thinsp;\u0026plusmn;\u0026thinsp;95%CI\u0026thinsp;=\u0026thinsp;3.1\u0026thinsp;\u0026plusmn;\u0026thinsp;1.7, p\u0026thinsp;\u0026lt;\u0026thinsp;0.001), with no significant difference in knowledge between Survey 2 and 3 (p\u0026thinsp;=\u0026thinsp;1.00).\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec11\"\u003e\n \u003ch2\u003eLevel 2 Learning \u0026ndash; Confidence in understanding and using PROMs\u003c/h2\u003e\n \u003cp\u003eThe related samples Friedman\u0026rsquo;s analysis of variance by ranks test (n\u0026thinsp;=\u0026thinsp;39) revealed no significant effect of time on clinician confidence in: 1) understanding what a PROM is (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;0.607, p\u0026thinsp;=\u0026thinsp;0.738); 2) understanding the significance of using PROMs (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;0.157, p\u0026thinsp;=\u0026thinsp;0.924); 3) recognising when to apply PROMs (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;0.157, p\u0026thinsp;=\u0026thinsp;0.924); 4) implementing PROMs (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;0.171, p\u0026thinsp;=\u0026thinsp;0.918); or 5) knowing what patient reported measures are available (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;0.639, p\u0026thinsp;=\u0026thinsp;0.726). The correlation between the questions \u0026alpha;\u0026thinsp;=\u0026thinsp;0.899. The mean ranks for each question at each time point are included in Table \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e.\u0026nbsp;\u003c/p\u003e\u0026nbsp;\u003ctable border=\"1\" id=\"Tab3\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eChange in participant confidence using patient reported outcome measures before and after receiving the education package\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey question\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey 1\u003c/p\u003e\n \u003cp\u003eMean rank\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey 2\u003c/p\u003e\n \u003cp\u003eMean rank\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey 3\u003c/p\u003e\n \u003cp\u003eMean rank\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow confident are you understanding what a patient reported outcome measure is?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.95\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.06\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.99\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow confident are you understanding the significance of patient reported outcome measures?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.08\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.96\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.96\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow confident are you recognising when to use patient reported outcome measures?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.04\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.99\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.97\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow confident are you implementing patient reported outcome measures?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.96\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow confident are you knowing what patient reported outcome measures are available?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.06\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec12\"\u003e\n \u003ch2\u003eLevel 3 \u0026ndash; Behaviour (Attitude towards PROMs)\u003c/h2\u003e\n \u003cp\u003eThere was no significant effect of time on how influential PROMs were to the treatment plan and patient management (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;3.644, p\u0026thinsp;=\u0026thinsp;0.162) (see Table \u003cspan class=\"InternalRef\"\u003e4\u003c/span\u003e). There was a significant effect of time on whether health professional should use PROMs using valid and reliable tools (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;6.982, p\u0026thinsp;=\u0026thinsp;0.03), however, the post-hoc Wilcoxon rank-tests revealed no significance difference between Survey 1 and 2 (Z\u0026thinsp;=\u0026thinsp;1.670, p\u0026thinsp;=\u0026thinsp;0.095) or between Survey 1 and 3 (Z\u0026thinsp;=\u0026thinsp;1.485, p\u0026thinsp;=\u0026thinsp;0.138). There was no effect of time on clinician attitudes to PROMs based on the remaining statements: 1) \u0026lsquo;PROMs enable you to get a better understanding of your patient\u0026rsquo;s progress\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;1.627, p\u0026thinsp;=\u0026thinsp;0.443); 2) \u0026lsquo;The use of validated PROMs is clinically helpful in an increasing medicolegal environment\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;1.853, p\u0026thinsp;=\u0026thinsp;0.396); 3) \u0026lsquo;The use of patient reported outcome measures could be helpful in justifying ongoing treatment to third parties\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;4.680, p\u0026thinsp;=\u0026thinsp;0.096); 4) \u0026lsquo;My patients are all different; therefore patient reported outcome measures would not be useful\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;0.628, p\u0026thinsp;=\u0026thinsp;0.731); 5) \u0026lsquo;Available patient reported outcome measures are unsuitable for the type of patients I treat\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;2.086, p\u0026thinsp;=\u0026thinsp;0.352); 6) \u0026lsquo;If I had more time, I would be interested in using patient reported outcome measures\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;0.609, p\u0026thinsp;=\u0026thinsp;0.738); 7) \u0026lsquo;I do not see the use of patient reported outcome measures as a priority\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;2.385, p\u0026thinsp;=\u0026thinsp;0.304); 8) \u0026lsquo;Patient reported outcome measures are unpopular with patients\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;1.681, p\u0026thinsp;=\u0026thinsp;0.432); 9) \u0026lsquo;Patient satisfaction is the most important outcome\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;4.978, p\u0026thinsp;=\u0026thinsp;0.083); 10) \u0026lsquo;I do not know enough about patient reported outcome measures to feel comfortable/confident using them\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;0.828, p\u0026thinsp;=\u0026thinsp;0.661); 11) \u0026lsquo;The patient discontinuing treatment puts me off using patient reported outcome measures\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;0.023, p\u0026thinsp;=\u0026thinsp;0.998); 12) \u0026lsquo;There is no need to change from the way that we have to assess/assessed patients\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;2.141, p\u0026thinsp;=\u0026thinsp;0.343); 13) \u0026lsquo;If I had to use patient reported outcome measures, I would prefer to choose which ones I used\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;2.049, p\u0026thinsp;=\u0026thinsp;0.359); 14) \u0026lsquo;Access to information about patient reported outcome measures is limited in my work environment\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;1.162, p\u0026thinsp;=\u0026thinsp;0.559); and 15) \u0026lsquo;It is not necessary to measure functional outcomes\u0026rsquo; (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;0.667, p\u0026thinsp;=\u0026thinsp;0.717). The correlation between the questions \u0026alpha;\u0026thinsp;=\u0026thinsp;0.655.\u003c/p\u003e\u0026nbsp;\u003ctable border=\"1\" id=\"Tab4\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 4\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eChange in participant attitude towards patient reported outcome measures before and after receiving the education package\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey question\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey 1\u003c/p\u003e\n \u003cp\u003eMean rank\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey 2\u003c/p\u003e\n \u003cp\u003eMean rank\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey 3\u003c/p\u003e\n \u003cp\u003eMean rank\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow influential are patient reported outcome measures to your treatment plan and patient management?\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.85\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.14\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHealth professionals should use patient reported outcome measures to monitor treatment outcomes using valid and reliable tools.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.81\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.17*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.03\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePatient reported outcome measures enable you to get a better understanding of your patient\u0026rsquo;s progress\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.94\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.95\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eThe use of validated patient reported outcome measures is clinically helpful in an increasing medicolegal environment.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.96\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eThe use of patient reported outcome measures could be helpful in justifying ongoing treatment to third parties\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.15\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.96\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.88\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMy patients are all different; therefore, patient reported outcome measures would not be useful\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.04\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.04\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.92\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAvailable patient reported outcome measures are unsuitable for the type of patients I treat\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.10\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.04\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.86\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eIf I had more time, I would be interested in using patient reported outcome measures\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.03\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eI do not see the use of patient reported outcome measures as a priority\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.14\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.91\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.95\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePatient reported outcome measures are unpopular with patients\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.96\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.91\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePatient satisfaction is the most important outcome\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.06\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.78\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.15\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eI do not know enough about patient reported outcome measures to feel comfortable/confident using them\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.00\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.08\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eThe patient discontinuing treatment puts me off using patient reported outcome measures\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.00\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.99\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.01\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eThere is no need to change from the way that we have to assess/assessed patients\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.86\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.08\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.06\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eIf I had to use patient reported outcome measures, I would prefer to choose which ones I used\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.97\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.90\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAccess to information about patient reported outcome measures is limited in my work environment\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.05\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.06\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.88\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv class=\"Section2\" id=\"Sec13\"\u003e\n \u003ch2\u003eLevel 3 \u0026ndash; Behaviour (Utilisation of PROMs)\u003c/h2\u003e\n \u003cp\u003eThere was no significant effect of time on the frequency of pain-related (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;0.192, p\u0026thinsp;=\u0026thinsp;0.909) or functional-related (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;3.69, p\u0026thinsp;=\u0026thinsp;0.158) PROM use (Table \u003cspan class=\"InternalRef\"\u003e5\u003c/span\u003e). There was, however, a significant effect of time on the frequency of health-related PROM use (\u0026chi;\u003csup\u003e2\u003c/sup\u003e (2)\u0026thinsp;=\u0026thinsp;8.310, p\u0026thinsp;=\u0026thinsp;0.016). Post-hoc Wilcoxon signed-rank tests revealed a significant increase in the mean rank score for health-related PROM use from Survey 1 (1.79) to Survey 3 (2.01) (Z\u0026thinsp;=\u0026thinsp;2.707, p\u0026thinsp;=\u0026thinsp;0.007, ES\u0026thinsp;=\u0026thinsp;0.41) with 14 positive differences, 3 negative differences, and 27 ties (Table \u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e). There was no difference in the mean rank score between Survey 1 to Survey 2 (Z\u0026thinsp;=\u0026thinsp;0.125, p\u0026thinsp;=\u0026thinsp;0.901), with 10 positive differences, 7 negative differences, and 35 ties. When asked about whether the educational package increased their use of patient reported outcome measures (PROMs) in clinical practice, 39% of participants in Survey 2 answered \u0026lsquo;yes\u0026rsquo; (41% No, 17% Unsure), compared to 56% in Survey 3 (28% No, 16% Unsure).\u0026nbsp;\u003c/p\u003e\u0026nbsp;\u003ctable border=\"1\" id=\"Tab5\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 5\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003eChanges in participant utilisation of patient reported outcome measures before and after receiving the education package\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey question\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey 1\u003c/p\u003e\n \u003cp\u003eMean rank\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey 2\u003c/p\u003e\n \u003cp\u003eMean rank\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSurvey 3\u003c/p\u003e\n \u003cp\u003eMean rank\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow often on average do you use PROMs in your everyday practice (ANY)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.03\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.00\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.97\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow often on average do you use PROMs in your everyday practice (PAIN)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.96\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.03\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow often on average do you use PROMs in your everyday practice (FUNCTIONAL)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.86\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.95\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.19\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eHow often on average do you use PROMs in your everyday practice (HEALTH)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.79\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.01\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2.19*\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003ctfoot\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"4\"\u003e*Statistically different to Survey 1 (p\u0026thinsp;\u0026lt;\u0026thinsp;0.05)\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tfoot\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis longitudinal study evaluated the impact of an educational package developed and implemented using the KTA framework, and which aimed to increase PROM knowledge and utilisation in a small cohort of chiropractors in Australia.\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003eThe results are presented and discussed according to three of the four levels of the NWKM \u0026ndash; reaction, learning, and behavior [32]. Overall, the reaction to the educational package was very positive, with most participants noting improved knowledge and a high level of satisfaction with the material. Although there was a significant improvement in knowledge after receiving the educational package, which was retained at 12 weeks, these changes were small and did not translate to improvements in confidence, attitude, or frequency of PROM usage. The findings suggest that changes in knowledge alone may be insufficient to influence the frequency of PROM utilisation by chiropractors.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eLevel 1 - Reaction\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eIt is important to evaluate the participant\u0026rsquo;s reactions to the course or training when evaluating the success of the educational intervention [33]. Overall, the findings confirmed a positive reaction to the educational package. Most respondents reported that the intervention was favourable, engaging, and relevant to their needs. These findings could therefore be important to create clinical behaviour change as Oreg, Vakola [34] found that organizational and behaviour change are not well implemented unless there is a positive reaction.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eSurvey 1 included items which allowed respondents to communicate directly with the content creators and inform design and content of the educational package [28, 35]. Importantly, 98% of those surveyed felt that the educational package they engaged with was highly relevant to their needs and matched their learning style; the latter of which has been associated with improved learning outcomes in educational [36]. Furthermore, 81% of the survey respondents were either somewhat or strongly satisfied with the overall educational package and 88% stated that the educational package was effective at increasing their knowledge and they would recommend it to their colleagues. Interestingly, only 70% of the respondents noted that they would recommend changes to their practice procedures after viewing the educational package. While research suggests that participant reaction to learning is important such measures may not correlate to how much participants learnt, or whether their behaviour changed [37-40]. For example, a recent study of acupuncturists noted that while the reaction to the training is important, levels 2 (learning) and 3 (behaviour) of the NWKM are the most relevant to determine the success of the program [41]. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eLevel 2 - Learning\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eLevel two of the NWKM is concerned with determining the degree of learning, knowledge, confidence, and skill acquisition because of training [26, 32, 42, 43]. To assess knowledge, evaluation of the participant\u0026rsquo;s recall, understanding and application of the learning should be measured [44].\u0026nbsp;Recall and understanding were assessed using three questions included in each survey and covered a range of topics including the definition of PROMs, categories of PROMs, and the identification of PROMs examples using multiple-choice questions. The results show a significant improvement in knowledge scores between survey 1 and survey 2, which were retained at survey 3, suggesting the intended learning outcomes were achieved and the educational package was successful [45, 46].\u0026nbsp;Nevertheless, the improvements in knowledge were modest\u0026nbsp;(~10%), and while statistically significant, may be insufficient to affect a change in participant behavior and attitude towards PROMs. Despite a recent survey of chiropractors in Australia suggesting that improving clinician understanding of PROMs and why/when to use PROMs would improve utilisation rates, our study, together with others, suggest that knowledge, in and of itself, may not be sufficient to change behavior [47, 48]. Similarly, confidence of the learner, which has been suggested to be a gap between learning and behaviour [49], was also not changed as a result of the educational package. We found no difference in confidence scores between survey 1, and surveys 2 and 3. Research suggests that motivation and confidence are key determinants to create behavioral change [50]. Boyce, Robertson [51] further suggest that building confidence is important if intentions are to be translated into behaviour change. Nevertheless, while confidence may bridge the gap between knowledge and behavior, improving knowledge alone seems insufficient to change confidence. Overall, this study suggests that the provision of an online educational package aimed to improve knowledge of PROMs in chiropractors had no effect on their confidence in understanding what PROMs were available or when and/or how to use them.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eLevel 3 - Behaviour\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eLevel 3 of the NWKM evaluates to what extent the newly acquired knowledge or skills have been practically applied [42, 52]. This level also assesses the degree to which the participants\u0026nbsp;apply what they learned and modified their behaviour based on the intervention [53]. In the current study we evaluated behavior change using questions related to two subthemes: 1) attitude toward PROMs; and 2) frequency of PROM utilisation. Participant\u0026rsquo;s attitude toward PROMs were evaluated using 16 questions, which covered a variety of attitudinal aspects relating to the importance and influence of PROMs in clinical practice. Of the 16 questions, only one question was significantly different after the provision of the educational package. Participants more strongly agreed with the statement: \u0026lsquo;Health professionals should use patient reported outcome measures to monitor treatment outcomes using valid and reliable tools\u0026rsquo; in survey 2, compared to survey 1. The educational package may have highlighted the arrival of the \u0026ldquo;era of accountability\u0026rdquo; with more pressure placed on health care providers to provide treatment evidence [54]. The last decade has seen an increasing focus and interest not only in a patients symptoms, but also in documenting the patient experience and their interactions with healthcare providers [55]. Nevertheless, the overall results do not support a change in clinician attitudes toward PROMs after receiving the educational package. Whilst difficult to determine, it is likely that respondents already had a very favorable view towards PROMs and so there may have been a ceiling effect limiting the impact of the educational package on participant attitudes. Previous studies do show that most chiropractors have a favorable view of PROMs [5, 15], however, these views do not necessarily correlate to higher levels of usage [15].\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAlthough 70% of the respondents reported that they would recommend changes to their practice procedures after viewing the educational package, the data did not support that those changes occurred. Furthermore, despite most respondents acknowledging the importance of PROMs and the need for clinical change to include PROMs, the clinical implementation did not increase after receiving the educational package. Except for health-related PROMs, the reported frequency of PROM utilisation did not change from survey 1 to surveys 2 and 3. We found a small but significant increase in the reported use of health-related PROMs from survey 1 to survey 3. Although there was a similar trend toward increased use of functional PROMs, these differences were not significant. These improvements may be explained, at least in part, by two factors. First, the current study, as well as previous studies [8], show that functional and health-related PROMs are less frequently used by chiropractors in clinical practice, compared to pain-related PROMs, and so their frequency of use may be more likely changed with an educational package that promotes awareness of these PROMs. Secondly, and further to this point, the educational package was specifically designed to address these practice gaps by providing a comprehensive summary of numerous health- and functional-related PROMs relevant to each body region. It is possible that much of this content may have been perceived as \u0026lsquo;new and novel\u0026rsquo; and therefore may be more likely to elicit a measurable change in behaviour, compared to the more commonly used pain-related PROMs. \u0026nbsp; \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eOverall, the educational package did not affect a measurable change in participant\u0026rsquo;s confidence, attitude, or frequency of PROM utilisation. Although we used the KTA framework to design the content of the educational package, we did not include any specific strategies to promote translation, but rather relied on knowledge to drive behavior change. Clearly, knowledge alone is not sufficient to affect behavioral change.\u0026nbsp;Although knowledge has been consistently identified as a barrier/facilitator to PROM utilisation in healthcare, multiple other barriers exist which were not addressed in the current study and may be more influential in changing behavior. Time was the most reported barrier to PROM usage by chiropractors [5,8]. Although the educational package provided links to source material there were no specific implementation strategies included in the package that may have eased the time burden of using PROMs in clinical practice. The provision of all relevant PROMs to chiropractic care, including background information, via a single electronic application may be a future strategy to better align PROM knowledge with PROM access, and reduce the time the burden of accessing PROMs in a clinical setting.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIt is possible that if this study had continued over a longer period of time, this may have increased the likelihood of measuring behavioral change. Axtell, Maitlis [56] found that the amount of learning transferred into practice one month post intervention was a strong predictor for learning transferred one year post intervention. According to Kirkpatrick D [57] the most accurate time to evaluate behaviour change is at least three months after the training was applied, although literature suggests that behaviour change may not occur until six months, to enable learners sufficient time to put their new skills into practice [58]. Therefore, a longer-term follow-up (up to 12 months) may have been needed to identify whether the short-term changes in knowledge changed behaviour.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eLimitations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn addition to the short follow-up period, there are several other limitations to this study. First, the low response rate to the cohort survey may limit the strength and generalisability of the findings. Although online surveys are a common research method, their success, particularly amongst health care professionals has been questioned [59]. Cunningham, Quan [59] imply that low survey response rates are common within the health care profession. Another limitation is bias, which is an inherent issue in the design of surveys [60]. Although the authors aimed to reduce selection bias and population bias by inviting all members of Australia\u0026rsquo;s main chiropractic associations to participate, an unintentional responder bias may have occurred. Due to the low response rate, the survey respondents may not be representative of the entire chiropractic profession in Australia. A further limitation of this study was that there was no control group. Additionally, given the substantial drop-out between survey 1 and survey 2, it is possible the participants who completed all three surveys may have already had favorable views of PROMs, and relatively high utilisation rates, and so there may have been a potential ceiling effect associated with the educational package that affected changes in confidence, attitude, and frequency of PROM usage.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eAn online educational package delivered to chiropractors in Australia was effective at improving knowledge of PROMs, including what PROMs are and why, how, and when they should be used in clinical practice. Despite modest improvements in knowledge, which were retained 3-months after the educational package was provided, there was no evidence that participants confidence, attitudes, or frequency of PROM use changed because of the intervention. This study suggests that knowledge of PROMs alone may be insufficient to change the frequency of their use in clinical practice by chiropractors in Australia.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cem\u003e\u003cstrong\u003eEthical Approval and Consent to participate\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eFull ethical approval was granted by the Human Research Ethics Committee of Central Queensland University, reference number 0000022391.\u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable \u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003e\u0026nbsp;Availability of data and materials\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable \u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare they have no competing interests \u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003e\u0026nbsp;Funding\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable \u003cem\u003e\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eAuthors\u0026apos; contributions\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eConcept development: NCC, SO, AGS. Design (planned the methods to generate the results): NCC, SO, AGS, GB. Ethics submission/approval NCC, AGS. Analysis and interpretation of data: SO, NCC, AGS, GB. Manuscript preparation NCC, SO, AGS, GB. Critical review: SO, AGS, GB. All authors read and approved the final manuscript.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable \u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cstrong\u003eAuthors\u0026apos; information(optional)\u003c/strong\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003csup\u003e1\u003c/sup\u003eCollege\u0026nbsp;of\u0026nbsp;Health\u0026nbsp;Sciences,\u0026nbsp;School\u0026nbsp;of\u0026nbsp;Health,\u0026nbsp;Medical\u0026nbsp;and\u0026nbsp;Applied\u0026nbsp;Sciences,\u0026nbsp;Central Queensland University, Branyan, QLD, Australia. \u003csup\u003e2\u003c/sup\u003eSchool of Applied Psychology, Griffith University, South Brisbane, QLD, Australia.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eSantana MJ, Haverman L, Absolom K, Takeuchi E, Feeny D, Grootenhuis M, et al. Training clinicians in how to use patient-reported outcome measures in routine clinical practice. Qual Life Res. 2015;24(7):1707-18. doi: 10.1007/s11136-014-0903-5\u003c/li\u003e\n\u003cli\u003eThompson L, Bekelman D, Allen L, Peterson P. Patient-Reported Outcomes in Heart Failure: Existing Measures and Future Uses. Current Heart Failure Reports. 2015;12(3):236-46. doi:10.1007/s11897-015-0253-9\u003c/li\u003e\n\u003cli\u003eWolpert M. Uses and Abuses of Patient Reported Outcome Measures (PROMs): Potential Iatrogenic Impact of PROMs Implementation and How It Can Be Mitigated. Administration and Policy in Mental Health and Mental Health Services Research. 2014;41(2):141-5. doi: 10.1007/s10488-013-0509-1\u003c/li\u003e\n\u003cli\u003eClohesy N, Schneiders A, Eaton S. Utilization of Low Back Pain Patient Reported Outcome Measures Within Chiropractic Literature: A Descriptive Review. Journal of Manipulative and Physiological Therapeutics. 2018;41(7):628-39. doi: https://doi.org/10.1016/j.jmpt.2017.11.007\u003c/li\u003e\n\u003cli\u003eHolmes M, Bishop F, Newell D, Field J, Lewith G. Chiropractors\u0026apos; views on the use of patient-reported outcome measures in clinical practice: a qualitative study. Chiropr Man Therap. 2018;26:50. doi: https://doi.org/10.1186/s12998-018-0219-6\u003c/li\u003e\n\u003cli\u003eWeldring T, Smith SMS. Patient-Reported Outcomes (PROs) and Patient-Reported Outcome Measures (PROMs). Health Serv Insights. 2013;6:61-8. doi: 10.4137/HSI.S11093\u003c/li\u003e\n\u003cli\u003eYeomans S, Liebenseon C. Applying outcome measures. JNMS. 1997;5.\u003c/li\u003e\n\u003cli\u003eClohesy N, Schneiders A. A preliminary investigation examining patient reported outcome measures for low back pain and utilisation amongst chiropractors in Australia: facilitators and barriers to clinical implementation. Chiropr Man Therap. 2018;26:38. doi: https://doi.org/10.1186/s12998-018-0208-9\u003c/li\u003e\n\u003cli\u003eBrinkman M, Barten D, Pisters M, Verheij R. Current use of PROMs and factors associated with their use in patients with nonspecific low back pain. Learning Health Systems. 2019;3(4):e10194. doi:10.1002/lrh2.10194\u003c/li\u003e\n\u003cli\u003eDuncan E, Murray J. The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review. BMC Health Services Research. 2012;12(1):96. doi: https://doi.org/10.1186/1472-6963-12-96\u003c/li\u003e\n\u003cli\u003eGreenhalgh J, Meadows K. The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. J Eval Clin Pract. 1999;5(4):401-16. doi: 10.1046/j.1365-2753.1999.00209.x.\u003c/li\u003e\n\u003cli\u003eAntunes B, Harding R, Higginson I. Implementing patient-reported outcome measures in palliative care clinical practice: a systematic review of facilitators and barriers. Palliat Med. 2014;28(2):158-75. doi: 10.1177/0269216313491619.\u003c/li\u003e\n\u003cli\u003eChapman-Smith D. The Role of Patient Questionnaires. The Chiropractic Report. 2012;26(5).\u003c/li\u003e\n\u003cli\u003eEilayyan O, Visca R, Zidarov D, Ware P, Bussi\u0026egrave;res A, Ahmed S. Developing theory-informed knowledge translation strategies to facilitate the use of patient-reported outcome measures in interdisciplinary low back pain clinical practices in Quebec: mixed methods study. BMC Health Services Research. 2020;20(1):789. doi: https://doi.org/10.1186/s12913-020-05616-5\u003c/li\u003e\n\u003cli\u003eClohesy N, Schneiders A, Barbery G, Obst S. Clinicians\u0026rsquo; perceived value and demographic factors that predict the utilisation of patient reported outcome measures for low back pain amongst chiropractors in Australia. Chiropr Man Therap. 2021;29(1):42. doi: https://doi.org/10.1186/s12998-021-00399-w\u003c/li\u003e\n\u003cli\u003eNilsson E, Orwelius L, Kristenson M. Patient-reported outcomes in the Swedish National Quality Registers. J Intern Med. 2016;279(2):141-53. doi: 10.1111/joim.12409\u003c/li\u003e\n\u003cli\u003eLynch E, Mudge A, Knowles S, Kitson A, Hunter S, Harvey G. \u0026quot;There is nothing so practical as a good theory\u0026quot;: a pragmatic guide for selecting theoretical approaches for implementation projects. BMC Health Serv Res. 2018;18(1):857. doi: https://doi.org/10.1186/s12913-018-3671-z\u003c/li\u003e\n\u003cli\u003eField B, Booth A, Ilott I, Gerrish K. Using the Knowledge to Action Framework in practice: a citation analysis and systematic review. Implementation Science. 2014;9(1):172. doi: https://doi.org/10.1186/s13012-014-0172-2\u003c/li\u003e\n\u003cli\u003eDavison C, Ndumbe-Eyoh C, C. Critical examination of knowledge to action models and implications for promoting health equity. International journal for equity in health. 2015;14(49). doi: https://doi.org/10.1186/s12939-015-0178-7\u003c/li\u003e\n\u003cli\u003eMoore J, Virva R, Henderson C, Lenca L, Butzer J, Lovell L, et al. Applying the Knowledge-to-Action Framework to Implement Gait and Balance Assessments in Inpatient Stroke Rehabilitation. Archives of Physical Medicine and Rehabilitation. 2020;103(7S):S230-54. doi: 10.1016/j.apmr.2020.10.133.\u003c/li\u003e\n\u003cli\u003eGraham I, Tetroe J. Some theoretical underpinnings of knowledge translation. Acad Emerg Med. 2007;14(11):936-41. doi: 10.1197/j.aem.2007.07.004.\u003c/li\u003e\n\u003cli\u003eGraham I, Logan J, Harrison M, Straus S, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13-24. doi: 10.1002/chp.47.\u003c/li\u003e\n\u003cli\u003eSinden K, MacDermid J. Does the knowledge-to-action (KTA) framework facilitate physical demands analysis development for firefighter injury management and return-to-work planning? J Occup Rehabil. 2014;24(1):146-59. doi: 10.1007/s10926-013-9442-0.\u003c/li\u003e\n\u003cli\u003eKirkpatrick W, Kirkpatrick J. Four levels of training evaluation. 2016. United States, ATD Press.\u003c/li\u003e\n\u003cli\u003eGill S, Sharma G. Evaluation of Vocational Training Program from the Trainees\u0026apos; Perspective: An Empirical Study. Pacific Business Review International. 2013;6(5):35-43.\u003c/li\u003e\n\u003cli\u003eRouse D. Employing Kirkpatrick\u0026apos;s evaluation framework to determine the effectiveness of health information management courses and programs. Perspect Health Inf Manag. 2011;8.\u003c/li\u003e\n\u003cli\u003eAHPRA. Chiropractic board statistics 2020. https://www.chiropracticboard.gov.au/About-the-Board/Annual-report.aspx. Accessed 25 March 2022.\u003c/li\u003e\n\u003cli\u003eBradwell P, Marr S. Making the most of collaboration: an international survey of public service co-design, Demos Report 23. London: 2008. \u003c/li\u003e\n\u003cli\u003eO\u0026apos;Cathain A, Thomas KJ. \u0026quot;Any other comments?\u0026quot; Open questions on questionnaires \u0026ndash; a bane or a bonus to research? BMC Medical Research Methodology. 2004;4(1):25. doi: https://doi.org/10.1186/1471-2288-4-25\u003c/li\u003e\n\u003cli\u003eSmith A, Ling P, Hill D. The Adoption of Multiple Modes of Delivery in Australian Universities. Journal of University Teaching \u0026amp; Learning Practice. 2006;3(2). doi: https://doi.org/10.53761/1.3.2.2\u003c/li\u003e\n\u003cli\u003eBland JM, Altman DG. Statistics notes: Cronbach\u0026apos;s alpha. BMJ. 1997;314(7080):572. doi: https://doi.org/10.1136/bmj.314.7080.572\u003c/li\u003e\n\u003cli\u003eCahapay M. International Journal of Assessment Tools in Education. Int J Assess Tools Educ. 2021;8(1):135\u0026ndash;44. doi:https://doi.org/10.21449/ijate.856143\u003c/li\u003e\n\u003cli\u003eG\u0026ouml;khan \u0026Ouml;. Program Evaluation through Kirkpatrick\u0026apos;s Framework. Pacific Business Review International. 2015;8(1).\u003c/li\u003e\n\u003cli\u003eOreg S, Vakola M, Armenakis A. Change Recipients\u0026rsquo; Reactions to Organizational Change: A 60-Year Review of Quantitative Studies. The Journal of Applied Behavioral Science. 2011;47(4):461-524. doi: https://doi.org/10.1177/0021886310396550\u003c/li\u003e\n\u003cli\u003eShahvari Z, Raisi F, Firoozi Khojastefar R, Yahyavi S, Mirsepassi Z. Memos on Sexuality Education Effectiveness; A Qualitative Study in Severe Mental Illness in Iran. International Journal of Sexual Health. 2020;32(3):254-66. doi: https://doi.org/10.1080/19317611.2020.1793047\u003c/li\u003e\n\u003cli\u003eBaykan Z, Na\u0026ccedil;ar M. Learning styles of first-year medical students attending Erciyes University in Kayseri, Turkey. Advances in Physiology Education. 2007;31(2):158-60. doi: 10.1152/advan.00043.2006\u003c/li\u003e\n\u003cli\u003eAlliger G, Janak E. Kirkpatrick\u0026apos;s levels of training criteria: Thirty years later. Personnel Psychology. 1989;42(2):331-42. doi: doi/abs/10.1111/j.1744-6570.1989.tb00661.x\u003c/li\u003e\n\u003cli\u003eArthur W, Bennett W, Edens P, Bell S. Effectiveness of Training in Organizations. Journal of Applied Psychology. 2003;88(2):234-45. doi: https://doi.org/10.1037/0021-9010.88.2.234\u003c/li\u003e\n\u003cli\u003eColquitt J, LePine J, Noe R. Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. Journal of Applied Psychology. 2000;85(5):678\u0026ndash;707. doi: 10.1037//0021-9010.85.5.678\u003c/li\u003e\n\u003cli\u003eTannenbaum SI, Yukl G. Training and development in work organizations. Annual Review of Psychology. 1992;43:399-441. doi: https://doi.org/10.1146/annurev.ps.43.020192.002151 \u003c/li\u003e\n\u003cli\u003eLiao S-C, Hsu S-Y. Evaluating A Continuing Medical Education Program: New World Kirkpatrick Model Approach. International Journal of Management, Economics and Social Sciences. 2019;8. doi:10.32327/IJMESS.8.4.2019.17\u003c/li\u003e\n\u003cli\u003ePhillips J, Phillips P. Handbook of Training Evaluation and Measurement Methods (4th). London: 2016. doi: https://doi.org/10.4324/9781315757230 \u003c/li\u003e\n\u003cli\u003eAllen LM, Hay M, Palermo C. Evaluation in health professions education\u0026mdash;Is measuring outcomes enough? Medical Education. 2022;56(1):127-36. doi: 10.1111/medu.14654 \u003c/li\u003e\n\u003cli\u003eKirkpatrick J, Kirkpatrick W. Introduction to the New World Kirkpatrick Model. 2019. https://www.kirkpatrickpartners.com/wp-content/uploads/2021/11/Introduction-to-the-Kirkpatrick-New-World-Model.pdf. Accessed on 6 January 2022. \u003c/li\u003e\n\u003cli\u003eDeodhar M, Powdwal S. Impact of continuing education programs (CEPs) on LIS professionals in academic libraries in Mumbai, India. Library management. 2017;38(2/3):117-30. \u003c/li\u003e\n\u003cli\u003eMohamed R, Ariyani A, Alias S. Evaluating the Effectiveness of a Training Program Using the Four Level Kirkpatrick Model in the Banking Sector in Malaysia: In: 3rd International Conference on Business and Economic Research, 12-13 March. Bandung, Indonesia2012. \u003c/li\u003e\n\u003cli\u003eArlinghaus KR, Johnston CA. Advocating for Behavior Change With Education. Am J Lifestyle Med. 2017;12(2):113-6. doi:10.1177/1559827617745479 \u003c/li\u003e\n\u003cli\u003eWorsley A. Nutrition knowledge and food consumption: can nutrition knowledge change food behaviour? Asia Pacific journal of clinical nutrition. 2002(11):579-85. doi: 10.1046/j.1440-6047.11.supp3.7.x. \u003c/li\u003e\n\u003cli\u003ePiryani RM, Dhungana GP, Piryani S, Sharma Neupane M. Evaluation of teachers training workshop at Kirkpatrick level 1 using retro-pre questionnaire. Adv Med Educ Pract. 2018;9:453-457. doi:10.2147/AMEP.S154166. \u003c/li\u003e\n\u003cli\u003eLubman D, Hall K, Gibbie T. Motivational interviewing techniques. Australian Journal for General Practitioners. 2012;41:660-7. \u003c/li\u003e\n\u003cli\u003eBoyce T, Robertson R, Dixon A. Commissioning and behaviour Change Kicking Bad Habits final report. 2008. \u003c/li\u003e\n\u003cli\u003eBaldwin T, Ford J. Transfer of Training: A Review and Directions of Future Research. Personnel Psychology. 1988;41(1):63\u0026ndash;105. doi: https://doi.org/10.1111/j.1744-6570.1988.tb00632.x \u003c/li\u003e\n\u003cli\u003eCheng E, Hampson I. Transfer of training: A review and new insights Journal of Management Reviews 2008;10(4):327\u0026ndash;41. doi: https://doi.org/10.1111/j.1468-2370.2007.00230.x \u003c/li\u003e\n\u003cli\u003eRelman A. Assessment and accountability : the third revolution in medical care. 2012. 87-90. doi: 10.1056/NEJM198811033191810. \u003c/li\u003e\n\u003cli\u003eDoyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1):e001570. doi:10.1136/bmjopen-2012-001570 \u003c/li\u003e\n\u003cli\u003eAxtell CM, Maitlis S, Yearta SK. Predicting immediate and longer-term transfer of training. Personnel review. 1997;26(3):201-13. doi: http://dx.doi.org/10.1108/00483489710161413 \u003c/li\u003e\n\u003cli\u003eKirkpatrick D. Great Ideas Revisited. Techniques for Evaluating Training Programs. Revisiting Kirkpatrick\u0026apos;s Four-Level Model. Training and Development. 1996;50(1). \u003c/li\u003e\n\u003cli\u003eMay G, Kahnweiler W. The effect of a mastery practiice design on learning and transfer in behavior modelling training Personnel psychology. 2000;53(2):353-73. doi: https://doi.org/10.1111/j.1744-6570.2000.tb00205.x \u003c/li\u003e\n\u003cli\u003eCunningham CT, Quan H, Hemmelgarn B, Noseworthy T, Beck CA, Dixon E, et al. Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol. 2015;15. doi:10.1186/s12874-015-0016-z \u003c/li\u003e\n\u003cli\u003eChoi B, Pak A. A catalog of biases in questionnaires. Prev Chronic Dis. 2005;2.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":true,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"chiropractic-and-manual-therapies","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"chmt","sideBox":"Learn more about [Chiropractic \u0026 Manual Therapies](http://chiromt.biomedcentral.com/)","snPcode":"12998","submissionUrl":"https://submission.springernature.com/new-submission/12998/3","title":"Chiropractic \u0026 Manual Therapies","twitterHandle":"@ChiroManTher","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Patient reported outcome measures, Chiropractic, Knowledge to Action, New World Kirkpatrick Model, Behaviour change ","lastPublishedDoi":"10.21203/rs.3.rs-1907999/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1907999/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eBackground: Patient Reported Outcome Measures (PROMs) have been shown to be valid and reliable indicators of health status and treatment outcomes, however, the current knowledge, understanding, and utilisation of PROMs within the Australian Chiropractic profession is limited. This study used the New World Kirkpatrick Model (NWKM) to evaluate whether an online PROM education package could improve knowledge, confidence, attitude, and utilisation of PROMs by chiropractors in Australia. \u003c/p\u003e\u003cp\u003eMethods: A longitudinal cohort interventional study of chiropractors in Australia. The recruitment phase and data collection period occurred from November 2020 – May 2021. Participants completed three online surveys two weeks before, four weeks after, and 12 weeks after receiving an online education package that included ten evidence-based region-specific modules on PROMs. Survey questions were grouped into five subthemes for analysis according to the NWKM levels: 1) Reaction; 2) Learning - knowledge; 3) Learning – confidence; 4) Behaviour – attitude; 5) and Behaviour - utilisation). \u003c/p\u003e\u003cp\u003eResults: Of the 113 participants that enrolled in the study, 43 completed all three survey and were included in the analysis. There was very positive to the education package with mean response scores (1-5 Likert scale) for the reaction questions ranging from 3.75 to 4.43. There was a small, but significant, increase in knowledge (out of 32) at four weeks (24.3 ± 6.1) and 12 weeks after receiving the education package (27.2 ± 5.5), compared to baseline (27.4 ± 5.1). There was no effect of intervention on clinician confidence or attitude towards PROMs. Utilisation of function- and pain-related PROMs did not change after the intervention. There was a small and significant (p\u0026lt;0.05) increase in utilisation of health-related PROMs 12 weeks after the intervention.\u003c/p\u003e\u003cp\u003eConclusion: Despite modest improvements in knowledge, which were retained 12 weeks after the educational package was provided, there was no evidence that participant confidence, attitude, or utilisation of PROMs changed because of the intervention. While the respondents’ have positive attitudes and beliefs regarding PROMs use, further education surrounding the clinical translation process into clinical practice is required.\u003c/p\u003e","manuscriptTitle":"The impact of a targeted education package on the knowledge, attitudes, and utilisation of patient reported outcome measures amongst chiropractors in Australia.","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-08-04 18:06:20","doi":"10.21203/rs.3.rs-1907999/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Minor Revision","date":"2022-08-29T02:31:08+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"","date":"2022-08-16T07:36:51+00:00","index":0,"fulltext":""},{"type":"reviewersInvited","content":"","date":"2022-08-03T06:03:51+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"Chiropractic \u0026 Manual Therapies","date":"2022-08-02T14:01:49+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2022-08-02T12:05:26+00:00","index":"","fulltext":""},{"type":"submitted","content":"Chiropractic \u0026 Manual Therapies","date":"2022-07-29T03:47:56+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"chiropractic-and-manual-therapies","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"chmt","sideBox":"Learn more about [Chiropractic \u0026 Manual Therapies](http://chiromt.biomedcentral.com/)","snPcode":"12998","submissionUrl":"https://submission.springernature.com/new-submission/12998/3","title":"Chiropractic \u0026 Manual Therapies","twitterHandle":"@ChiroManTher","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2187903c-d4b9-4d71-ae43-9b5c34307da6","owner":[],"postedDate":"August 4th, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2022-09-21T08:42:56+00:00","versionOfRecord":[],"versionCreatedAt":"2022-08-04 18:06:20","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-1907999","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1907999","identity":"rs-1907999","version":["v1"]},"buildId":"7rjqhiLT3MXkJMwkYKINL","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}