Assessment during clinical education among nursing students using two different assessment instruments.

Assessment during clinical education among nursing students using two different assessment instruments. | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Assessment during clinical education among nursing students using two different assessment instruments. Tomas Nilsson, Italo Masiello, Eva Broberger, Veronica Lindström This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3920417/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 07 Aug, 2024 Read the published version in BMC Medical Education → Version 1 posted 11 You are reading this latest preprint version Abstract Background: Assessment of undergraduate students using assessment instruments in the clinical setting is known to be complex. The aim of this study was therefore to examine whether two different assessment instruments, containing learning objectives (LO`s) with similar content, results in similar assessments by the clinical supervisors and to explore clinical supervisors’ experiences of assessment regarding the two different assessment instruments. Method: A mixed-methods approach was used. Four simulated care encounter scenarios were evaluated by 50 supervisors using two different assessment instruments. 28 follow-up interviews were conducted. Descriptive statistics and logistic binary regression were used for quantitative data analysis, along with qualitative thematic analysis of interview data. Result: While significant differences were observed within the assessment instruments, the differences were consistent between the two instruments, indicating that the quality of the assessment instruments were considered equivalent. Supervisors noted that the relationship between the students and supervisors could introduce subjectivity in the assessments and that working in groups of supervisors could be advantageous. In terms of formative assessments, the Likert scale was considered a useful tool for evaluating learning objectives. However, supervisors had different views on grading scales and the need for clear definitions. The supervisors concluded that a complicated assessment instrument led to limited very-day usage and did not facilitate formative feedback. Furthermore, supervisors discussed how their experiences influenced the use of the assessment instruments, which resulted in different descriptions of the experience. These differences led to a discussion of the need of supervisor teams to enhance the validity of assessments. Conclusion: The findings showed that there were no significant differences in pass/fail gradings using the two different assessment instruments. The quantitative data suggests that supervisors struggled with subjectivity, phrasing, and definitions of the LO´s and the scales used in both instruments. This resulted in arbitrary assessments that were time-consuming and resulted in limited usage in the day-to-day assessment. To mitigate the subjectivity, supervisors suggested working in teams and conducting multiple assessments over time to increase assessment validity. 1. Introduction During undergraduate studies to become a registered nurse (RN), the assessment of clinical competence includes an assessment of both theoretical knowledge and practical skills [28]. The importance of high-quality clinical education for students that provides constructive and adapted feedback is a key factor affecting the student’s learning process and should not be understated [28, 30] . Historically, different assessment instruments have been designed aiming both to support the clinical supervisors and to provide standardised, fair assessments of students’ achievement of the learning objectives (LOs)[10] . However, the assessment of nursing students’ skills and competence using assessment instruments is known to be complex and affected by several varied factors [29]. The assessment instrument, Assessment of Clinical Education (ACIEd), that is used in this study context, is used in a summative fashion to verify that the students earn a passing grade during clinical education. Formative assessment aims to support the students through continuous feedback. To support the student’s development towards becoming a RN, an assessment instrument that facilitates both summative and formative assessment is needed [1]. 2. Background A well-educated RN is essential to achieve good health care. To realise this, supervisors, as well as students, need support during clinical education in facilitating student learning [12]. Nevertheless, supporting, and supervising students is known to be complex since learning in the clinical setting is affected by several factors, such as the clinical context, the student’s own strengths, workplace challenges, expectations, and the student’s social network and prior knowledge and skills [3, 4]. For the students to succeed in their learning, an un-bias assessment with LO`s that are relevant for the student’s progression is necessary. 2.1 Assessment in clinical education Assessment is a vital part of clinical education and should include both skills and theoretical knowledge reflecting the requirements of the RN everyday work and having a close connection to the university curriculum [17]. The assessment of nursing students can be accomplished in a formative or summative way. The formative assessment is meant to guide the nursing students in their learning progress and is a joint discussion between the student and the supervisor where strengths and areas of improvement are identified and addressed [6]. The summative assessment’s prime objective is to ensure that LOs are achieved, and it commonly occurs on one or two occasions during a period/session of clinical education with a university teacher present [15]. However, clinical assessment is sometimes hampered by the clinical supervisors’ lack of training in assessing students’ nursing skills and competence and/or the supervisors’ lack of competence in using a standard assessment instrument [17]. In addition, a heavy workload and staff shortages in the clinical setting make it difficult for the clinical supervisors to find time to assess students properly. It is also known that the relationship between supervisor and student can affect the assessment [11]. To aid the assessment of students’ nursing skills and competence, assessment instruments need to be reliable, valid, easy to use and adapted to the clinical setting [7]. Today, several different assessment instruments exist which all have strengths and weaknesses and focus on various aspects of the student’s learning. These instruments utilize different approaches to assessment where for example Observed Structural Clinical Examination (OSCE) uses checklists and Assessment of Clinical Education (ACIEd) uses more complex LO`s that require subjective assessments [26, 27, 22]. ACIEd, is used by several universities in Sweden for assessment of LO`s during clinical education. The ACIEd was designed for a mid and final assessment, where the mid assessment results in: “In line with expected achievement” or “Plan of action is needed”. The final assessment results in a “Pass” or “Fail” grade on each LO. In clinical education, every LO needs to be passed to get a final pass grade [26, 27]. Criticism has been raised in the ambulance services in Stockholm, Sweden, towards ACIEd, claiming that it is ill adapted for daily use and has a summative character where progression is hard to visualize. Therefore, an assessment instrument was developed from the ACIEd and named the Ambulance Assessment Instrument (AAI). The AAI was constructed with more distinct and phrased LOs listed one by one, instead of using complex LOs with several sub-LOs imbedded in one. A seven graded Likert scale facilitated formative assessment. The rationale for clarifying LOs was that earlier research has shown that complex LOs lead to interpretations and inconsistencies in assessment [17]. Therefore, this was considered when the AAI was developed, resulting in one-sentence LOs without sub-criteria. The assessment instruments can be found in appendix 1. However, to incorporate a new assessment instrument in clinical education, it needs to be validated. In this case the research group choose to validate the developed AAI instrument against the existing instrument ACIEd. The aim of this study was therefore to examine whether two different assessment instruments, containing LO´s with similar content, results in similar assessments by the clinical supervisors and to explore clinical supervisors’ experiences of assessment regarding the two different assessment instruments. 2.3 Clinical education in the ambulance service The clinical setting in this study was the ambulance services in Stockholm, Sweden. Care in an ambulance is characterised by short patient encounters, ever-changing environments, and patients seeking care for all varieties of complaints [19, 25, 31]. Typical is also a lack of preparation time and little or no chance for the students to pause and step out of the care encounter to discuss strategies or reflect on care situations that have occurred. The ambulance service has similar challenges to those faced in ambulatory care [9]. Historically, the ambulance service is an uncommon placement for clinical education in undergraduate nursing education due to the environment and lack of RNs working in the service. At present, every ambulance in Sweden is staffed by at least one RN [16], and it is stated that nursing students have the possibility to learn nursing skills in the ambulance services [17, 18, 20, 21]. In the region of Stockholm, the nurses have at least one year of additional training which, for example, could be prehospital care, anesthesiology, emergency care, psychiatric care, or midwifery. The amount of work experience can vary greatly. Among the employed nurses its almost an even split between males and females. 3. Material and Method A mixed-method study design was used [23]. Quantitative data was generated from supervisors when assessing pre-recorded, simulated patient encounters performed by students. Individual interviews, conducted directly after the assessments, were completed in accordance with a mixed-method design. 3.1 Assessment instruments The ACIEd used in this study consists of five LOs with sub-criteria concerning what the student needs to achieve to receive a Pass grade during a course in emergency care. The LOs are designed to cover all aspects of the six-week clinical education in relation to the course objectives resulting in LOs with several goals embedded in one, for example, LO 1, which translated reads: 1. Approach and support patients and their relatives in respectful consultation and perform nursing care based on the patient's experience of the situation. To clarify this LO and reduce the risk of inconsistencies in assessment one-sentence LOs without sub-criteria was reconstructed in the AAI, and one example of this reads as follows: 1.1 To what extent was the patient treated with respect? 1.2 To what extent was the patient allowed to describe his or her situation? 1.3 To what extent did the student create a safe care situation? The complete list of LO´s used can be found in appendix one. To enable formative assessment by displaying progress using the assumption that students’ performances will generate higher grades as their clinical education progresses, the AAI has a seven-point Likert scale with a pass grade marked as 5, meaning that scores 1 to 4 results in a fail grade. The Likert scale was given descriptions from 1 = “ Not at all” to 7 = “ To a great extent”. Furthermore, the ACIEd separates midterm assessment from final assessment, where grades in midterm are referred to as “In line with objective” or “Plan of action is required”. The final assessment using ACIEd results in a pass or fail grade. The AAI provides a formative assessment, which is meant to be repeated frequently but can also act as a basis for grading in an equivalent way as the ACIEd. 3.2 Simulated patient encounters Four different recorded simulated scenarios were used in the study (recorded time: 6min 58 sec, 3min 14sec, 5min 58sec, 2min 32 sec). The scenarios had variations of student performances with the intention of generating variations in supervisor assessments. Differences were seen in both students’ assessments of the patient and in their treatment strategies. Two scenarios included a student interacting with a standardised patient, while two scenarios had a student interacting with a patient simulator manikin. The standardised patient was a middle-aged woman with fatigue and dizziness. The patient simulator manikin was a young man with abdominal pain. The nursing students participating in the simulation scenarios were recruited from the fifth semester of the nursing study programme at one university in Stockholm, Sweden, after they had finished their six-week clinical education in the ambulance services and all grades had been made official. Both male and female students participated in the simulations. 3.3 Participants A convenience sample of 50 clinical supervisors from the ambulance service, participated in the study, having varying experience of supervisorship. The participants were recruited at three different emergency departments. By selecting different emergency departments, it was possible to include participants with different ambulance service employers (private and public). The clinical supervisors from the ambulance service were asked to participate in the study after they arrived at the emergency department and the hand-over of the patient was completed. The convenience sampling of clinical supervisors was used due to the difficulty in recruiting participants at the ambulance station, as they were constantly mobile during their shifts. No participants were excluded due to experience, gender, educational level, or other factors. 3.4 Data collection In total, 50 clinical supervisors from the ambulance service assessed and graded four recorded simulations in accordance with the LO described in the ACIEd and the AAI instrument. No supervisor declined the offer to participate in this part of the study. Before the assessment started, a randomisation process was conducted. Firstly, the order of the simulations was randomised using a lottery (standardised patient vs. patient simulator manikin.). Secondly, the two scenarios were randomised using lottery (scenarios 1 and 2). The recorded simulations were watched in one sequence without time to reflect or discuss the assessment and grading with others. The qualitative data consisted of interviews, conducted after the participants had assessed the simulated patient encounters. The interviews started with the open question “What are your thoughts about the assessment instruments?” Probing questions were then used to explore the participants’ experience of using the assessment instruments when grading the student's performance in the recorded simulations. The interviews varied in length from a few minutes up to 30 minutes. In total, 28 interviews were conducted (Female: 11 Male: 17). Fieldnotes were used during the interviews and after every finished interview the fieldnotes were summarized and reviewed. Theses reviews were used to make changes to the probing questions in relations to the aim of the study. In 22 cases there were no interviews conducted due to participants’ shortage of time. 3.5 Data analysis The data was analysed in two parts, a quantitative and qualitative. The quantitative data was analysed by compiling the assessments generated by the two assessment instruments and compared in a simple figure where the difference was described from the perspective of how many pass grades the assessment instrument generated. Secondly, a logistic binary regression analysis was used to examine whether the grades generated by two different assessment instruments was affected by the supervisor’s gender and/or experience. To explore whether work experience as an RN affected the grading of student’s performance, a dichotomisation of the work experience variable was carried out. The variables were dichotomised into \(\le\) 6 and 6 > years of working experience as RNs (Dichotomised 1 for \(\le\) 6 and 2 for 6 >). This dichotomisation was based on the theory that experienced nurses can use their experience, knowledge, and additional perspectives instead of relying on standardised guidelines, tests, and regulations [2] to assess the students according to the LO`s. A gender dichotomisation was also performed to investigate if there was any difference in grading related to gender which was defined as male or female with no consideration to other gender definitions (Gender was coded 1 for males and 2 for females). This dichotomisation was based on the diversity of the staff in the ambulance service. The independent variables for both instruments were coded as 1 for a pass grade and 2 for fail. After considering the number of included supervisors in the study the p value was set to 0.05. The data was analysed using Statistical Package for the Social Sciences, version 24, Chicago, IL, USA in combination with Microsoft Excel 2010 (Microsoft Corp, Richmond, WA, USA). The interviews were analysed using a thematic analysis approach [5]. The method was chosen due to its flexible nature. The themes constructed was done in a “theory driven way” meaning that the research question was clearly present in the coding in contrast to an inductive analysis approach. First, in the analysis the field notes were read several times to gain familiarity with the content. Secondly, codes were identified that described clinical supervisors’ experience of using the two different assessment instruments were identified. Thirdly, the codes were examined, and by identifying broader patterns of meaning, potential themes were constructed. Fourthly, a thematic map was constructed, and the themes were checked on two levels. First the codes were checked against the theme making sure that the codes formed a coherent pattern. In the second level the themes were checked against the entire dataset and in relations to the other themes to ensure that the themes did not intertwine with each other and finally, the themes were named. In the fifth step the themes and sub-themes were related back to the narrative making sure that the themes captured the full story and that each theme was unique and contained valid information. The “story” that the themes and sub-themes were checked against, was the researcher’s contextual knowledge and experience. Lastly, the report was written up using the themes and the sub- themes. 4. Quantitative Results In total, 34 (68%) male and 16 (32%) female clinical supervisors participated in the study. The work experience as an RN ranged from four months to 19 years, with a mean of 7.95 years. The work experience from the ambulance services ranged from 2.5 months to 35 years with a mean of 6.63 years (missing data n=5). All participants had experience of clinical supervision and assessing nursing students during clinical education. The logistic binary regression analysis showed that there was no significant difference in 23 out of 24 LOs as displayed in table 1-4. One significant difference was found in the ACIEd LO 2 (p= 0.021) when the length of experience among the clinical supervisors was used as a dependent variable (table 4). LO 2 assessed the students’ knowledge of the technical equipment and how they interacted with the patient while using the equipment. Table 1 E ffect of the dependent variable ‘ Genders’ on grading related to learning objectives using a manikin for both the AAI and ACIEd with 1 degree of freedom. Independent variable LOs Beta coefficient Standard error Wald chi-square test P-Value Exponentiation of the B coefficient AAI* LO1 .451 .866 .271 .602 1.571 AAI LO 2 -.932 .813 1.314 .252 .394 AAI LO 3 .277 1.024 .073 .787 1.319 ACIEd** LO 1 -.478 .960 .247 .619 .620 ACIEd LO 2 .969 .965 1.009 .315 2.636 ACIEd LO 3 -.003 .965 .000 .998 .997 *AAI – Ambulance Assessment Instrument **ACIEd – Assessment of Clinical Education Table 2 E ffect of the dependent variable ‘ Experience’ on grading related to learning objectives using a manikin for both the AAI and ACIEd with 1 degree of freedom. Independent variable LOs Beta coefficient Standard error Wald chi- square test P-Value Exponentiation of the B coefficient AAI* LO 1 .600 1.117 .288 .591 1.822 AAI LO 2 2.418 1.279 3.574 .059 11.219 AAI LO 3 .167 1.306 .016 .898 1.182 ACIEd** LO 1 1.346 1.186 1.288 .256 3.841 ACIEd LO 2 -2.193 1.546 2.012 .156 .112 ACIEd LO 3 -1.955 1.220 2.568 .109 .142 *AAI – Ambulance Assessment Instrument **ACIEd – Assessment of Clinical Education Table 3 E ffect of the dependent variable ‘ Gender’ on grading related to learning objectives using the Standardized patient for both the AAI and ACIEd with 1 degree of freedom. Independent variable LOs Beta coefficient Standard error Wald chi-square test P-Value Exponentiation of the B coefficient AAI* LO 1 .499 1.021 .239 .625 1.647 AAI LO 2 .561 .597 .883 .347 1.752 AAI LO 3 .540 .825 .428 .513 1.716 ACIEd** LO 1 -.992 .987 1.010 .315 .371 ACIEd LO 2 .328 .725 .205 .651 1.388 ACIEd LO 3 -.329 1.092 .091 .763 .720 *AAI – Ambulance Assessment Instrument **ACIEd – Assessment of Clinical Education Table 4 E ffect of the dependent variable ‘ Experience’ on grading related to Learning objectives using the Standardized patient for both the AAI and ACIEd with 1 degree of freedom. Independent variable LOs Beta coefficient Standard error Wald chi-square test P-Value Exponentiation of the B coefficient AAI* LO 1 -.405 .970 .174 .676 .667 AAI LO 2 .662 .562 1.388 .239 1.939 AAI LO 3 -.527 .788 .447 .504 .590 ACIEd** LO 1 -.204 .972 .044 .834 .815 ACIEd LO 2 1.918 .834 5.289 .021* 6.805 ACIEd LO 3 -1.260 1.160 1.181 .277 .284 *AAI – Ambulance Assessment Instrument **ACIEd – Assessment of Clinical Education 4.1 Qualitative Results The thematic analysis used to explore clinical supervisors’ experience of assessment in relation to the two different assessment instruments resulted in three themes: Learning objectives , Assessment and Supervisorship. 4.1.1 Learning objectives Supervisors described a variety of experiences relating to the LO`s in the two sub-themes: Phrasing of the LOs, subjectivity in the LOs, Complexity of the LO`s. The subthemes describe the supervisors’ view of the learning objectives in relation to how the LO`s are constructed and how they are interpreted as well as how they are used in the daily activities. Phrasing of the learning objectives: The supervisors discussed the language used in the ACIEd to describe the LO`s and that the academic writing created confusion and was complex and hard to understand. They also discussed the lack of clear definitions, and difficulties using the ACIEd with several sub-criteria embedded in one LO, which resulted in individual interpretation of the LO and a risk of subjective assessment. According to the supervisors’ reasoning, several sub-criteria in the same LO complicated the assessment when students performed well according to some of the sub-criteria and poorly according to others. The supervisors expressed that the LOs used in AAI had a clearer phrasing making it easier to use. LO`s related to communication and patient relations were considered complicated to assess using the ACIEd in contrast to LO`s regarding medical procedures where right and wrong was clearly defined. Medical procedures were considered easier to assess in both instruments. Complexity of the learning objective When assessing the students’ nursing and care skills in the simulations, the supervisors claimed that the complexity of ACIEd made it challenging to explaining why they assessed as they did. In the clinic setting the supervisors said that the complexity of the LO`s made the usage of ACIEd time-consuming and poorly adapted to everyday work, which resulted in limited use, and only used in a summative way right before mid and end assessments. Altogether, the ACIEd was considered by the supervisors to be time-consuming, and leading to subjective interpretations and sometimes, conflicting assessments. The AAI was considered more relevant due to the “simplified” LO`s. However, AAI was thought to be difficult to use due to the Likert scale ranging from 1-7 were lack of clear definitions of the grading steps increased the complexity of assessing the LO`s. 4.1.2 Assessment Supervisors described assessment from different perspectives as presented below in the following sub-themes: Summative Vs Formative assessment, Subjectivity in assessment, Pass/Fail or Likert scale, and Supervisors experience related to assessments. Within the sub-themes there were variations in how the supervisors described their experiences. Summative Vs Formative assessment The supervisors discussed if and how the assessment instruments could be used for formative or summative assessment and concluded that both instruments could be used for both purposes. However, the supervisors discussed whether ACIEd was harder to use as a formative instrument due to the sub-criteria in the LO`s and that the pass/fail scale were less useful then the 7 graded Likert scale when it came to displaying progress. The supervisors discussed whether displaying the student’s progression had a positive pedagogical value and they concluded as preferable relating it to formative feedback. Supervisors also discussed the frequency of formative assessments. Some supervisors raised concerns related to the increased workload generated by daily documented, formative feedback. Subjectivity in assessment To decrease the subjectivity, the supervisors discussed involving the patients in the assessments of students’ performance. They argued that the only persons who could assess the LOs concerning the patients’ own perception were the patients themselves. The supervisors concluded that more care encounters, with assessments between every encounter would probably produce a more accurate assessment of the student’s nursing skills. The supervisors also discussed the risk that the students would only be assessed according to the supervisor’s interpretation of what is a pass performance in relation to the assessment instrument. The supervisors found that the assessments with a Likert scale could be beneficial but that the grading steps needed to be carefully defined to avoid subjective assessments. The definitions could be made clearer by using examples in relations to the grading steps and the LO`s where requirements could be listed for each step. Supervisors said that assessments using a Pass/Fail grading was too definite. Supervisors described that assessing the care encounters became complicated since student’s performances may contain good and bad performances and with Pass/Fail grading the nuances did not become clear. Supervisors experience related to assessments. Supervisors’ description of their relationship to the LO`s varied where some had more issues than others. Supervisors with less experience were more critical towards the LO`s then experienced supervisors. Furthermore, supervisors with less experience discussed that the student were obligated to display skills and knowledge and that supervisors were obligated to assess in accordance with the LO`s, meaning responsibility for demonstrating knowledge and skills fell on the student. The more experienced supervisors argued that the LO`s were more like guidelines than specific goals to achieve, and that they used LO`s as a basis for discussions with the students. They argued that their clinical experience and understanding were the basis for the assessment, making the challenge with the LO`s less important, which contrasted with the view of the less experienced supervisors who interpreted the LO´s more literally. Furthermore, the more experienced supervisors discussed that complex care encounters offered scarce opportunities for the students to display knowledge and skills and that several care encounters were needed to assess knowledge and skills over time. 4.1.3 Supervisorship The supervisorship was described by the supervisors in the following two sub-themes: Relationship with the students and Teams of supervisors. Relationship with the students All supervisors said that a relationship with the student could be a confounder when assessing students. They discussed whether a good relationship with the student would probably result in a more favourable assessment. The supervisors also claimed that a troubled relationship with the student might result in disinterest from their perspective which could result in diminishing feedback and lowered clinical education quality resulting in higher risk for failure. The supervisors discussed that formative feedback could help detect a lack of progression at an early stage and that measures could be taken to improve the situation for the student. Among the female supervisors a recurring statement was that it was difficult to assess the student negatively in the simulations because it felt harsh or even cruel to fail a student. Among the male supervisor’s similar feelings were described, but they related to the fact that assessments are not carried out on single occasions but over time, making the assessment instrument unfit for this kind of assessment. Male supervisors argued to a greater extent that it was hard to assess the student in the simulation due to lack of information about both patients and students. They argued that although some simulations were not as described in “textbook” examples, guidelines were bent daily and therefore it would be unfair to expect “textbook” care from the students. Hence, the male supervisors argued that the assessment instrument was a tool used for discussion and could not stand alone which it did in the simulations. Teams of supervisors The supervisors highlighted continuity as something important for the student’s learning, but it could also complicate their assessment since they may develop a relationship with the student. The desire for continuity in supervisorship could also create a problem when working schedules changed. This could result in the involvement of other supervisors, and disruption in the individual learning plan. The supervisors with longer experienced argued that it could be favourable with more than one supervisor involved in the assessment of the student, due to different perspectives. They argued that a supervisor team with different combinations of knowledge could be beneficial for the student, but that such teams must be coordinated and documented to ensure that the student’s learning progress was not hindered. 5. Discussion The findings showed that there were no significant differences in pass/fail gradings using the two different assessment instruments with the same content in the LOs, meaning that no matter what instrument was used the grade was the same when supervisors assessed the students in the simulated scenarios. However, there were significant differences within the assessment instruments, but the differences were consistent between the instruments, meaning that the quality of the assessment was considered equal. The differences within the assessment instrument can be explained by supervisor bias. Chong et al showed in their study that seniority was a source for bias in LO`s related to communication but that the bias did not persist in LO`s related to physical examinations which is in line with the findings in this study [8]. The interviews with the clinical supervisors provided a more vivid picture of the complexity of assessments. Firstly, the intention of the assessments must be made clear for everyone. If the objective is formative assessment, the data indicates that using Likert scales is preferable to display progression. The complexity, phrasing and definition were recurring in the interviews, and it is a worth discussing why this is an issue. Supervisors rarely had any training in how to use the assessment instruments and as described, the more experience supervisors did not excel in their knowledge of the instruments but used their clinical experience to assess students from their own perspective. With this logic it would be wise to invest in training the supervisors rather than simplifying the LO´s. Prior research has shown that support from the supervisors during the clinical training is crucial to create a positive learning environment and to improve assessments [31]. However, the need for supervisor training and support does not mean that improving the assessment instrument is unnecessary. If a Likert scale is used, a clear definition of the grading steps is important as well as the layout of the instrument. As described by Immonen et al, it is important that the instrument is adapted to the every-day work, and fast and easy to use without losing its reliability and validity [14]. Subjectivity and complexity were reoccurring statements during the interviews. The supervisors highlighted this in all aspects regarding the clinical education. The supervisors’ own experience’ and expectations play a vital part in the clinical education as well as the relationship between the supervisor and the students which is supported in prior research [24]. To decrease subjectivity, an increase in the number of assessments made by more than one supervisor could be beneficial. Assessments after every care encounter in the ambulance service would generate a good basis for an overall assessment of the students’ performance. By organising the supervisors in teams, the different knowledges of the supervisors could be effectively utilized and possibly decrease subjectivity resulting in an improvement of the validity of the assessment. In contrast to this, prior research has shown that the relationship between the student and the supervisor is important to build trust and thereby a positive learning environment [30]. With documented, formative assessments accessible for both students and supervisors, continuity could be created through communication between supervisors. By using digital devices with LO`s and a Likert scale prepared in a mobile application, the assessments could be made easier to access, faster to use and the results could be displayed as a progression curve visible for students and supervisors. Digitalization of the assessment instrument also holds advantages concerning student possibility to argue for their grades. Without documented progress the students are in the hands of the supervisors as the only source of information about their performances. To further strengthen the validity of the assessments, other sources of feedback could be used. The supervisors discussed involving patients in the assessments and concluded that this could reduce subjectivity in the assessments and add other perspectives to the assessment. Further research is needed to fully understand the complexity of assessments and what methods to use to improve the quality of the assessment and strengthen the students learning. 5.1 Methodological considerations There are several limitations to this study. The number of supervisors included in the study was decided in discussion with the research team and a statistician after considering the availability of clinical supervisors and the extent of data needed for analysis. No power calculations were performed. Using standardised patients and simulation for assessing and grading students’ ability to care for patients may not reflect the clinical reality. However, since every care encounter is unique it would have been difficult to conduct a similar study in a clinical setting. Another limitation concerns the qualitative data collection. The interviewer is well known in the ambulance service, and this may have affected how the participants’ discussions concerning their experiences of using the assessment instruments in both a positive and negative way and increase the risk of bias. Conducting interviews between ambulance missions may also be considered as a limitation since the time for interviews was limited which could have resulted in participants not being able to develop their reasoning. The interviews were documented through field notes, and important information could have been missed. Field notes limit the possibility of quotations, which could have strengthened the validity of the findings in the interviews. 6. Conclusion The findings showed that there were no significant differences in pass/fail gradings using the two different assessment instruments containing the same LOs. However, the qualitative data suggests that supervisors struggled with subjectivity in the assessments as well as phrasing and definitions of the LO´s and the scales used in both instruments. This resulted in arbitrary assessments that were time-consuming and resulted in limited usage in the day-to-day assessment. The supervisors argued that the AAI was better adapted for formative assessment due to its Likert scale and simplified LOs, but a clear definition of the grading scales was considered important. Further research is needed concerning the validity of the assessments and how teams of supervisors can utilize different perspectives to improve the quality of the assessments. Digitalization could play a vital role in documenting feedback from multiple sources to enhance the formative feedback given to students during their clinical training. More research is needed to fully understand the mechanism behind the subjectivity of assessments and what methods could be used to strengthen the quality of the assessments and improve the quality of the clinical education. Abbreviations AAI Ambulance Assessment Instrument ACIEd Assessment of Clinical Education LO Learning objectives OSCE Observed Structural Clinical Examination RN Registered nurses Declarations Ethics approval and consent to practice A written, informed consent was sign by all participants before participation of the study. The study was designed according to the ethical principles for research described by the International Council of Nurses [13]. The study was approved by the Swedish Ethical Review Authority, Stockholm, Sweden (2016/594-31). All methods were performed in accordance with the relevant guidelines and regulations stipulated in the Helsinkideclaration. Consent to publication No individual data or other sources of data requiring consent for publication were used in this article. Availability of data and materials The dataset generated and analysed in this study is available from the corresponding author on reasonable request. Competing interests The authors reported no competing financial or non-financial interests. All authors contributed significantly to the content of this article. Funding Funding for this study was provided by the Karolinska Institute. Author contributions All authors contributed significantly to the study design in terms of the choice of analytic methods and interview design. All authors played an active role in reading and providing feedback about the material throughout the entire process. TN, EB and VL were involved in carrying out group interviews of the students and contributed significantly by reading and providing feedback about the manuscript. All authors reviewed and approved the submitted manuscript. Acknowledgements I’ve would like to send a special thanks to my family for their never wavering support and to all participants how made this paper possible. I also wanted to extend my gratitude’s towards Ambulansen I StorStockholm AB for facilitating my studies and this paper. References Arrogante, O., González-Romero, G. M., López-Torre, E. M., Carrión-García, L., & Polo, A. (2021). Comparing formative and summative simulation-based assessment in undergraduate nursing students: nursing competency acquisition and clinical simulation satisfaction. BMC Nurs, 20 (1), 92. doi:10.1186/s12912-021-00614-2 Benner, P. (1982). From novice to expert. Am J Nurs, 82 (3), 402-407. Berkhout, J. J., Helmich, E., Teunissen, P. W., van den Berg, J. W., van der Vleuten, C. P., & Jaarsma, A. D. (2015). Exploring the factors influencing clinical students' self-regulated learning. Med Educ, 49 (6), 589-600. doi:10.1111/medu.12671 Berkhout, J. J., Helmich, E., Teunissen, P. W., van der Vleuten, C. P. M., & Jaarsma, A. D. C. (2018). Context matters when striving to promote active and lifelong learning in medical education. Med Educ, 52 (1), 34-44. doi:10.1111/medu.13463 Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3 (2), 77-101. doi:10.1191/1478088706qp063oa Brown, J. M., Lowe, K., Fillingham, J., Murphy, P. N., Bamforth, M., & Shaw, N. J. (2014). An investigation into the use of multi-source feedback (MSF) as a work-based assessment tool. Med Teach, 36 (11), 997-1004. doi:10.3109/0142159x.2014.909920 Burke, E., Kelly, M., Byrne, E., Ui Chiardha, T., Mc Nicholas, M., & Montgomery, A. (2016). Preceptors' experiences of using a competence assessment tool to assess undergraduate nursing students. Nurse Educ Pract, 17 , 8-14. doi:10.1016/j.nepr.2016.01.004 Chong, L., Taylor, S., Haywood, M., Adelstein, B. A., & Shulruf, B. (2018). Examiner seniority and experience are associated with bias when scoring communication, but not examination, skills in objective structured clinical examinations in Australia. J Educ Eval Health Prof, 15 , 17. doi:10.3352/jeehp.2018.15.17 Coburn, C. V., Gilland, D., Owen, M., & Amar, A. (2018). Ambulatory care education: Preparing nurses for the future of healthcare. Nurse Educ Today, 66 , 79-81. doi:10.1016/j.nedt.2018.03.015 Fluit, C., Bolhuis, S., Grol, R., Ham, M., Feskens, R., Laan, R., & Wensing, M. (2012). Evaluation and feedback for effective clinical teaching in postgraduate medical education: validation of an assessment instrument incorporating the CanMEDS roles. Med Teach, 34 (11), 893-901. doi:10.3109/0142159x.2012.699114 Franklin, N., & Melville, P. (2015). Competency assessment tools: An exploration of the pedagogical issues facing competency assessment for nurses in the clinical environment. Collegian, 22 (1), 25-31. Gibbons, S. W., Adamo, G., Padden, D., Ricciardi, R., Graziano, M., Levine, E., & Hawkins, R. (2002). Clinical evaluation in advanced practice nursing education: using standardized patients in Health Assessment. J Nurs Educ, 41 (5), 215-221. ICN. (2012). The ICN Code of ethics for nurses. Immonen, K., Oikarainen, A., Tomietto, M., Kääriäinen, M., Tuomikoski, A. M., Kaučič, B. M., . . . Mikkonen, K. (2019). Assessment of nursing students' competence in clinical practice: A systematic review of reviews. Int J Nurs Stud, 100 , 103414. doi:10.1016/j.ijnurstu.2019.103414 Kibble, J. D. (2017). Best practices in summative assessment. Adv Physiol Educ, 41 (1), 110-119. doi:10.1152/advan.00116.2016 Lindström, V., Bohm, K., & Kurland, L. (2015). Prehospital care in Sweden. Notfall + Rettungsmedizin, 18 (2), 107-109. doi:10.1007/s10049-015-1989-1 McCarthy, B., & Murphy, S. (2008). Assessing undergraduate nursing students in clinical practice: do preceptors use assessment strategies? Nurse Educ Today, 28 (3), 301-313. doi:10.1016/j.nedt.2007.06.002 Melby, V. (2000). Experiential learning in pre-hospital emergency care: a qualitative study. Nurse Educ Today, 20 (8), 638-645. doi:10.1054/nedt.2000.0496 Nilsson, T., & Lindstrom, V. (2016). Clinical decision-making described by Swedish prehospital emergency care nurse students - An exploratory study. Int Emerg Nurs, 27 , 46-50. doi:10.1016/j.ienj.2015.10.006 Nilsson, T., & Lindstrom, V. (2017). Nursing students' perceptions of learning nursing skills in the ambulance service. Nurse Educ Pract, 24 , 1-5. doi:10.1016/j.nepr.2017.02.011 Nyqvist, J., Brolin, K., Nilsson, T., & Lindström, V. (2020). The learning environment and supportive supervision promote learning and are based on the relationship between students and supervisors - A qualitative study. Nurse Educ Pract, 42 , 102692. doi:10.1016/j.nepr.2019.102692 Patrício, M. F., Julião, M., Fareleira, F., & Carneiro, A. V. (2013). Is the OSCE a feasible tool to assess competencies in undergraduate medical education? Med Teach, 35 (6), 503-514. doi:10.3109/0142159x.2013.774330 Schoonenboom, J., & Johnson, R. B. (2017). How to Construct a Mixed Methods Research Design. Kolner Z Soz Sozpsychol, 69 (Suppl 2), 107-131. doi:10.1007/s11577-017-0454-1 Sundstrom, B. W., & Dahlberg, K. (2012). Being prepared for the unprepared: a phenomenology field study of Swedish prehospital care. J Emerg Nurs, 38 (6), 571-577. doi:10.1016/j.jen.2011.09.003 Ten Cate, O., & Regehr, G. (2019). The Power of Subjectivity in the Assessment of Medical Trainees. Acad Med, 94 (3), 333-337. doi:10.1097/acm.0000000000002495 Ulfvarson, J., & Oxelmark, L. (2012). Developing an assessment tool for intended learning outcomes in clinical practice for nursing students. Nurse Education Today, 32 (6), 703-708. doi:https://doi.org/10.1016/j.nedt.2011.09.010 Ulfvarson, J., Oxelmark, L., & Jirwe, M. (2018). Assessment in clinical education: A comparison between a generic instrument and a course-specific criterion-based instrument. Nordic Journal of Nursing Research , 205715851876840. doi:10.1177/2057158518768404 van de Ridder, J. M., Stokking, K. M., McGaghie, W. C., & ten Cate, O. T. (2008). What is feedback in clinical education? Med Educ, 42 (2), 189-197. doi:10.1111/j.1365-2923.2007.02973.x Van Horn, E., & Christman, J. (2017). Assessment of Nursing Student Confidence Using the Clinical Skills Self-Efficacy Scale. Nurs Educ Perspect, 38 (6), 344-346. doi:10.1097/01.Nep.0000000000000169 Wallin, K., Fridlund, B., & Thorén, A. B. (2013). Prehospital Emergency Nursing students' experiences of learning during prehospital clinical placements. Int Emerg Nurs, 21 (3), 197-203. doi:10.1016/j.ienj.2012.09.003 Wallin, K., Hörberg, U., Harstäde, C. W., Elmqvist, C., & Bremer, A. (2020). Preceptors´ experiences of student supervision in the emergency medical services: A qualitative interview study. Nurse Educ Today, 84 , 104223. doi:10.1016/j.nedt.2019.104223 Wireklint Sundstrom, B., Bremer, A., Lindstrom, V., & Vicente, V. (2018). Caring science research in the ambulance services: an integrative systematic review. Scand J Caring Sci . doi:10.1111/scs.12607 Additional Declarations No competing interests reported. Supplementary Files Appendix1.docx Cite Share Download PDF Status: Published Journal Publication published 07 Aug, 2024 Read the published version in BMC Medical Education → Version 1 posted Editorial decision: Revision requested 24 May, 2024 Reviews received at journal 24 May, 2024 Reviewers agreed at journal 08 May, 2024 Reviewers agreed at journal 24 Mar, 2024 Reviews received at journal 05 Mar, 2024 Reviewers agreed at journal 25 Feb, 2024 Reviewers invited by journal 24 Feb, 2024 Editor assigned by journal 24 Feb, 2024 Editor invited by journal 21 Feb, 2024 Submission checks completed at journal 21 Feb, 2024 First submitted to journal 02 Feb, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3920417","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":274180171,"identity":"32aa6d9c-63b3-4b0d-a67b-19767167beab","order_by":0,"name":"Tomas Nilsson","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA0ElEQVRIiWNgGAWjYBAC+wMQmocPTFUk8BDUwgaleSCMMyRogTAY2xIIO4yNvTvx4w+GOzJs/McfPi6clybDwH/4AH4tPGc3S/MwPONhk8gxNp65LYeHQSINv1VsErkbpBkYDgO18LBJ826rAGrhMSCkZfPPHyAt/MefSfPOAWrhP/+BkJZtEjwgLQwJZtK8DUCHMeTg1QHyyzZrHoPDEL/wHEsDMtIIOIy9d/PNHxWH7flBIcZTkwxkHH6A3xowQDaWDaeqUTAKRsEoGAVEAwD2ADWAxtBwrQAAAABJRU5ErkJggg==","orcid":"","institution":"Department of Clinical Science and Education, Södersjukhuset, Karolinska Institutet","correspondingAuthor":true,"prefix":"","firstName":"Tomas","middleName":"","lastName":"Nilsson","suffix":""},{"id":274180172,"identity":"dea179f7-8f31-4cfe-9e0a-c1b74c721222","order_by":1,"name":"Italo Masiello","email":"","orcid":"","institution":"Linnaeus University","correspondingAuthor":false,"prefix":"","firstName":"Italo","middleName":"","lastName":"Masiello","suffix":""},{"id":274180173,"identity":"2b009513-13f1-4d0c-b6ab-f03e86c62ba3","order_by":2,"name":"Eva Broberger","email":"","orcid":"","institution":"Department for Neurobiology, Care Sciences, and Society, Division of Nursing, Karolinska Institutet","correspondingAuthor":false,"prefix":"","firstName":"Eva","middleName":"","lastName":"Broberger","suffix":""},{"id":274180174,"identity":"7f40b162-e5ac-4106-8f94-052a78c77758","order_by":3,"name":"Veronica Lindström","email":"","orcid":"","institution":"Umeå University","correspondingAuthor":false,"prefix":"","firstName":"Veronica","middleName":"","lastName":"Lindström","suffix":""}],"badges":[],"createdAt":"2024-02-02 10:00:30","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3920417/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3920417/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12909-024-05771-x","type":"published","date":"2024-08-07T15:57:52+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":62298918,"identity":"78c59f30-6476-45f2-afb1-617303a71c47","added_by":"auto","created_at":"2024-08-12 16:17:27","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":654569,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3920417/v1/2dc1398d-2156-4746-bbdf-ddbb3f8778d6.pdf"},{"id":51522576,"identity":"7acfcc45-4be4-4156-a438-9049b6b38405","added_by":"auto","created_at":"2024-02-23 04:22:40","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":20327,"visible":true,"origin":"","legend":"","description":"","filename":"Appendix1.docx","url":"https://assets-eu.researchsquare.com/files/rs-3920417/v1/8b701b557594e6ac3632c72f.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Assessment during clinical education among nursing students using two different assessment instruments.","fulltext":[{"header":"1.\tIntroduction ","content":"\u003cp\u003eDuring undergraduate studies to become a registered nurse (RN), the assessment of clinical competence includes an assessment of both theoretical knowledge and practical skills [28]. The importance of high-quality clinical education for students that provides constructive and adapted feedback is a key factor affecting the student\u0026rsquo;s learning process and should not be understated [28, 30] . Historically, different assessment instruments have been designed aiming both to support the clinical supervisors and to provide standardised, fair assessments of students\u0026rsquo; achievement of the learning objectives (LOs)[10] . However, the assessment of nursing students\u0026rsquo; skills and competence using assessment instruments is known to be complex and affected by several varied factors [29]. The assessment instrument, Assessment of Clinical Education (ACIEd), that is used in this study context, is used in a summative fashion to verify that the students earn a passing grade during clinical education. Formative assessment aims to support the students through continuous feedback. To support the student\u0026rsquo;s development towards becoming a RN, an assessment instrument that facilitates both summative and formative assessment is needed [1].\u0026nbsp;\u003c/p\u003e"},{"header":"2. Background","content":"\u003cp\u003eA well-educated RN is essential to achieve good health care. To realise this, supervisors, as well as students, need support during clinical education in facilitating student learning [12]. Nevertheless, supporting, and supervising students is known to be complex since learning in the clinical setting is affected by several factors, such as the clinical context, the student\u0026rsquo;s own strengths, workplace challenges, expectations, and the student\u0026rsquo;s social network and prior knowledge and skills [3, 4]. For the students to succeed in their learning, an un-bias assessment with LO`s that are relevant for the student\u0026rsquo;s progression is necessary.\u003c/p\u003e \u003cdiv id=\"Sec2\" class=\"Section2\"\u003e \u003ch2\u003e2.1 Assessment in clinical education\u003c/h2\u003e \u003cp\u003eAssessment is a vital part of clinical education and should include both skills and theoretical knowledge reflecting the requirements of the RN everyday work and having a close connection to the university curriculum [17]. The assessment of nursing students can be accomplished in a formative or summative way. The formative assessment is meant to guide the nursing students in their learning progress and is a joint discussion between the student and the supervisor where strengths and areas of improvement are identified and addressed [6]. The summative assessment\u0026rsquo;s prime objective is to ensure that LOs are achieved, and it commonly occurs on one or two occasions during a period/session of clinical education with a university teacher present [15]. However, clinical assessment is sometimes hampered by the clinical supervisors\u0026rsquo; lack of training in assessing students\u0026rsquo; nursing skills and competence and/or the supervisors\u0026rsquo; lack of competence in using a standard assessment instrument [17]. In addition, a heavy workload and staff shortages in the clinical setting make it difficult for the clinical supervisors to find time to assess students properly. It is also known that the relationship between supervisor and student can affect the assessment [11]. To aid the assessment of students\u0026rsquo; nursing skills and competence, assessment instruments need to be reliable, valid, easy to use and adapted to the clinical setting [7]. Today, several different assessment instruments exist which all have strengths and weaknesses and focus on various aspects of the student\u0026rsquo;s learning. These instruments utilize different approaches to assessment where for example Observed Structural Clinical Examination (OSCE) uses checklists and Assessment of Clinical Education (ACIEd) uses more complex LO`s that require subjective assessments [26, 27, 22]. ACIEd, is used by several universities in Sweden for assessment of LO`s during clinical education. The ACIEd was designed for a mid and final assessment, where the mid assessment results in: \u0026ldquo;In line with expected achievement\u0026rdquo; or \u0026ldquo;Plan of action is needed\u0026rdquo;. The final assessment results in a \u0026ldquo;Pass\u0026rdquo; or \u0026ldquo;Fail\u0026rdquo; grade on each LO. In clinical education, every LO needs to be passed to get a final pass grade [26, 27]. Criticism has been raised in the ambulance services in Stockholm, Sweden, towards ACIEd, claiming that it is ill adapted for daily use and has a summative character where progression is hard to visualize. Therefore, an assessment instrument was developed from the ACIEd and named the Ambulance Assessment Instrument (AAI). The AAI was constructed with more distinct and phrased LOs listed one by one, instead of using complex LOs with several sub-LOs imbedded in one. A seven graded Likert scale facilitated formative assessment. The rationale for clarifying LOs was that earlier research has shown that complex LOs lead to interpretations and inconsistencies in assessment [17]. Therefore, this was considered when the AAI was developed, resulting in one-sentence LOs without sub-criteria. The assessment instruments can be found in appendix 1. However, to incorporate a new assessment instrument in clinical education, it needs to be validated. In this case the research group choose to validate the developed AAI instrument against the existing instrument ACIEd. The aim of this study was therefore to examine whether two different assessment instruments, containing LO\u0026acute;s with similar content, results in similar assessments by the clinical supervisors and to explore clinical supervisors\u0026rsquo; experiences of assessment regarding the two different assessment instruments.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.3 Clinical education in the ambulance service\u003c/h2\u003e \u003cp\u003eThe clinical setting in this study was the ambulance services in Stockholm, Sweden. Care in an ambulance is characterised by short patient encounters, ever-changing environments, and patients seeking care for all varieties of complaints [19, 25, 31]. Typical is also a lack of preparation time and little or no chance for the students to pause and step out of the care encounter to discuss strategies or reflect on care situations that have occurred. The ambulance service has similar challenges to those faced in ambulatory care [9]. Historically, the ambulance service is an uncommon placement for clinical education in undergraduate nursing education due to the environment and lack of RNs working in the service. At present, every ambulance in Sweden is staffed by at least one RN [16], and it is stated that nursing students have the possibility to learn nursing skills in the ambulance services [17, 18, 20, 21]. In the region of Stockholm, the nurses have at least one year of additional training which, for example, could be prehospital care, anesthesiology, emergency care, psychiatric care, or midwifery. The amount of work experience can vary greatly. Among the employed nurses its almost an even split between males and females.\u003c/p\u003e \u003c/div\u003e"},{"header":"3. Material and Method","content":"\u003cp\u003eA mixed-method study design was used [23]. Quantitative data was generated from supervisors when assessing pre-recorded, simulated patient encounters performed by students. Individual interviews, conducted directly after the assessments, were completed in accordance with a mixed-method design.\u003c/p\u003e\n\u003cdiv id=\"Sec5\" class=\"Section2\"\u003e\n \u003ch2\u003e3.1 Assessment instruments\u003c/h2\u003e\n \u003cp\u003eThe ACIEd used in this study consists of five LOs with sub-criteria concerning what the student needs to achieve to receive a Pass grade during a course in emergency care. The LOs are designed to cover all aspects of the six-week clinical education in relation to the course objectives resulting in LOs with several goals embedded in one, for example, LO 1, which translated reads:\u003c/p\u003e\u003cspan\u003e\n \u003cp\u003e\u003cem\u003e1. Approach and support patients and their relatives in respectful consultation and perform nursing care based on the patient\u0026apos;s experience of the situation.\u003c/em\u003e\u003c/p\u003e\n \u003c/span\u003e\n \u003cp\u003eTo clarify this LO and reduce the risk of inconsistencies in assessment one-sentence LOs without sub-criteria was reconstructed in the AAI, and one example of this reads as follows:\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e1.1 To what extent was the patient treated with respect?\u003c/em\u003e\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e1.2 To what extent was the patient allowed to describe his or her situation?\u003c/em\u003e\u003c/p\u003e\n \u003cp\u003e\u003cem\u003e1.3 To what extent did the student create a safe care situation?\u003c/em\u003e\u003c/p\u003e\n \u003cp\u003eThe complete list of LO\u0026acute;s used can be found in appendix one. To enable formative assessment by displaying progress using the assumption that students\u0026rsquo; performances will generate higher grades as their clinical education progresses, the AAI has a seven-point Likert scale with a pass grade marked as 5, meaning that scores 1 to 4 results in a fail grade. The Likert scale was given descriptions from 1 = \u0026ldquo;\u003cem\u003eNot at all\u0026rdquo;\u003c/em\u003e to 7 = \u0026ldquo;\u003cem\u003eTo a great extent\u0026rdquo;.\u003c/em\u003e Furthermore, the ACIEd separates midterm assessment from final assessment, where grades in midterm are referred to as \u0026ldquo;In line with objective\u0026rdquo; or \u0026ldquo;Plan of action is required\u0026rdquo;. The final assessment using ACIEd results in a pass or fail grade. The AAI provides a formative assessment, which is meant to be repeated frequently but can also act as a basis for grading in an equivalent way as the ACIEd.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e\n \u003ch2\u003e3.2 Simulated patient encounters\u003c/h2\u003e\n \u003cp\u003eFour different recorded simulated scenarios were used in the study (recorded time: 6min 58 sec, 3min 14sec, 5min 58sec, 2min 32 sec). The scenarios had variations of student performances with the intention of generating variations in supervisor assessments. Differences were seen in both students\u0026rsquo; assessments of the patient and in their treatment strategies. Two scenarios included a student interacting with a standardised patient, while two scenarios had a student interacting with a patient simulator manikin. The standardised patient was a middle-aged woman with fatigue and dizziness. The patient simulator manikin was a young man with abdominal pain. The nursing students participating in the simulation scenarios were recruited from the fifth semester of the nursing study programme at one university in Stockholm, Sweden, after they had finished their six-week clinical education in the ambulance services and all grades had been made official. Both male and female students participated in the simulations.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e\n \u003ch2\u003e3.3 Participants\u003c/h2\u003e\n \u003cp\u003eA convenience sample of 50 clinical supervisors from the ambulance service, participated in the study, having varying experience of supervisorship. The participants were recruited at three different emergency departments. By selecting different emergency departments, it was possible to include participants with different ambulance service employers (private and public). The clinical supervisors from the ambulance service were asked to participate in the study after they arrived at the emergency department and the hand-over of the patient was completed. The convenience sampling of clinical supervisors was used due to the difficulty in recruiting participants at the ambulance station, as they were constantly mobile during their shifts. No participants were excluded due to experience, gender, educational level, or other factors.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e\n \u003ch2\u003e3.4 Data collection\u003c/h2\u003e\n \u003cp\u003eIn total, 50 clinical supervisors from the ambulance service assessed and graded four recorded simulations in accordance with the LO described in the ACIEd and the AAI instrument. No supervisor declined the offer to participate in this part of the study. Before the assessment started, a randomisation process was conducted. Firstly, the order of the simulations was randomised using a lottery (standardised patient vs. patient simulator manikin.). Secondly, the two scenarios were randomised using lottery (scenarios 1 and 2). The recorded simulations were watched in one sequence without time to reflect or discuss the assessment and grading with others.\u003c/p\u003e\n \u003cp\u003eThe qualitative data consisted of interviews, conducted after the participants had assessed the simulated patient encounters. The interviews started with the open question \u0026ldquo;What are your thoughts about the assessment instruments?\u0026rdquo; Probing questions were then used to explore the participants\u0026rsquo; experience of using the assessment instruments when grading the student\u0026apos;s performance in the recorded simulations. The interviews varied in length from a few minutes up to 30 minutes. In total, 28 interviews were conducted (Female: 11 Male: 17). Fieldnotes were used during the interviews and after every finished interview the fieldnotes were summarized and reviewed. Theses reviews were used to make changes to the probing questions in relations to the aim of the study. In 22 cases there were no interviews conducted due to participants\u0026rsquo; shortage of time.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec9\" class=\"Section2\"\u003e\n \u003ch2\u003e3.5 Data analysis\u003c/h2\u003e\n \u003cp\u003eThe data was analysed in two parts, a quantitative and qualitative. The quantitative data was analysed by compiling the assessments generated by the two assessment instruments and compared in a simple figure where the difference was described from the perspective of how many pass grades the assessment instrument generated. Secondly, a logistic binary regression analysis was used to examine whether the grades generated by two different assessment instruments was affected by the supervisor\u0026rsquo;s gender and/or experience. To explore whether work experience as an RN affected the grading of student\u0026rsquo;s performance, a dichotomisation of the work experience variable was carried out. The variables were dichotomised into \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\le\\)\u003c/span\u003e\u003c/span\u003e6 and 6 \u0026gt; years of working experience as RNs (Dichotomised 1 for \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\le\\)\u003c/span\u003e\u003c/span\u003e6 and 2 for 6 \u0026gt;). This dichotomisation was based on the theory that experienced nurses can use their experience, knowledge, and additional perspectives instead of relying on standardised guidelines, tests, and regulations [2] to assess the students according to the LO`s. A gender dichotomisation was also performed to investigate if there was any difference in grading related to gender which was defined as male or female with no consideration to other gender definitions (Gender was coded 1 for males and 2 for females). This dichotomisation was based on the diversity of the staff in the ambulance service. The independent variables for both instruments were coded as 1 for a pass grade and 2 for fail. After considering the number of included supervisors in the study the p value was set to 0.05. The data was analysed using Statistical Package for the Social Sciences, version 24, Chicago, IL, USA in combination with Microsoft Excel 2010 (Microsoft Corp, Richmond, WA, USA).\u003c/p\u003e\n \u003cp\u003eThe interviews were analysed using a thematic analysis approach [5]. The method was chosen due to its flexible nature. The themes constructed was done in a \u0026ldquo;theory driven way\u0026rdquo; meaning that the research question was clearly present in the coding in contrast to an inductive analysis approach. First, in the analysis the field notes were read several times to gain familiarity with the content. Secondly, codes were identified that described clinical supervisors\u0026rsquo; experience of using the two different assessment instruments were identified. Thirdly, the codes were examined, and by identifying broader patterns of meaning, potential themes were constructed. Fourthly, a thematic map was constructed, and the themes were checked on two levels. First the codes were checked against the theme making sure that the codes formed a coherent pattern. In the second level the themes were checked against the entire dataset and in relations to the other themes to ensure that the themes did not intertwine with each other and finally, the themes were named.\u003c/p\u003e\n \u003cp\u003eIn the fifth step the themes and sub-themes were related back to the narrative making sure that the themes captured the full story and that each theme was unique and contained valid information. The \u0026ldquo;story\u0026rdquo; that the themes and sub-themes were checked against, was the researcher\u0026rsquo;s contextual knowledge and experience. Lastly, the report was written up using the themes and the sub- themes.\u003c/p\u003e\n\u003c/div\u003e"},{"header":"4. Quantitative Results","content":"\u003cp\u003eIn total, 34 (68%) male and 16 (32%) female clinical supervisors participated in the study. The work experience as an RN ranged from four months to 19 years, with a mean of 7.95 years. The work experience from the ambulance services ranged from 2.5 months to 35 years with a mean of 6.63 years (missing data n=5). All participants had experience of clinical supervision and assessing nursing students during clinical education. The logistic binary regression analysis showed that there was no significant difference in 23 out of 24 LOs as displayed in table 1-4. One significant difference was found in the ACIEd LO 2 (p= 0.021) when the length of experience among the clinical supervisors was used as a dependent variable (table 4). LO 2 assessed the students\u0026rsquo; knowledge of the technical equipment and how they interacted with the patient while using the equipment. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 1 \u003cem\u003eE\u003c/em\u003e\u003cem\u003effect of the dependent variable\u003c/em\u003e \u0026lsquo;\u003cem\u003eGenders\u0026rsquo; on grading related to learning objectives using a manikin for both the\u003c/em\u003e \u003cem\u003eAAI and ACIEd with 1 degree of freedom.\u003c/em\u003e\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003eIndependent variable LOs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.236706689536879%\" valign=\"top\"\u003e\n \u003cp\u003eBeta coefficient\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003eStandard error\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003eWald chi-square test\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.437392795883362%\" valign=\"top\"\u003e\n \u003cp\u003eP-Value\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003eExponentiation of the B coefficient\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003eAAI* LO1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.236706689536879%\" valign=\"top\"\u003e\n \u003cp\u003e.451\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003e.866\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003e.271\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.437392795883362%\" valign=\"top\"\u003e\n \u003cp\u003e.602\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003e1.571\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003eAAI LO 2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.236706689536879%\" valign=\"top\"\u003e\n \u003cp\u003e-.932\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003e.813\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003e1.314\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.437392795883362%\" valign=\"top\"\u003e\n \u003cp\u003e.252\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003e.394\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003eAAI LO 3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.236706689536879%\" valign=\"top\"\u003e\n \u003cp\u003e.277\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003e1.024\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003e.073\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.437392795883362%\" valign=\"top\"\u003e\n \u003cp\u003e.787\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003e1.319\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003eACIEd** LO 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.236706689536879%\" valign=\"top\"\u003e\n \u003cp\u003e-.478\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003e.960\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003e.247\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.437392795883362%\" valign=\"top\"\u003e\n \u003cp\u003e.619\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003e.620\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003eACIEd LO 2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.236706689536879%\" valign=\"top\"\u003e\n \u003cp\u003e.969\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003e.965\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003e1.009\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.437392795883362%\" valign=\"top\"\u003e\n \u003cp\u003e.315\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003e2.636\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003eACIEd LO 3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"14.236706689536879%\" valign=\"top\"\u003e\n \u003cp\u003e-.003\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003e.965\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.780445969125214%\" valign=\"top\"\u003e\n \u003cp\u003e.000\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.437392795883362%\" valign=\"top\"\u003e\n \u003cp\u003e.998\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"19.382504288164665%\" valign=\"top\"\u003e\n \u003cp\u003e.997\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e*AAI \u0026ndash; Ambulance Assessment Instrument\u003c/p\u003e\n\u003cp\u003e**ACIEd \u0026ndash; Assessment of Clinical Education\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 2 \u003cem\u003eE\u003c/em\u003e\u003cem\u003effect of the dependent variable\u003c/em\u003e \u0026lsquo;\u003cem\u003eExperience\u0026rsquo; on grading related to learning objectives using a manikin for both the AAI and ACIEd with 1 degree of freedom.\u003c/em\u003e\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"582\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"21.305841924398624%\" valign=\"top\"\u003e\n \u003cp\u003eIndependent variable LOs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.807560137457045%\" valign=\"top\"\u003e\n \u003cp\u003eBeta coefficient\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.542955326460481%\" valign=\"top\"\u003e\n \u003cp\u003eStandard error\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.72852233676976%\" valign=\"top\"\u003e\n \u003cp\u003eWald chi- square test\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"7.903780068728523%\" valign=\"top\"\u003e\n \u003cp\u003eP-Value\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.711340206185568%\" valign=\"top\"\u003e\n \u003cp\u003eExponentiation of the B coefficient\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"21.305841924398624%\" valign=\"top\"\u003e\n \u003cp\u003eAAI* LO 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.807560137457045%\" valign=\"top\"\u003e\n \u003cp\u003e.600\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.542955326460481%\" valign=\"top\"\u003e\n \u003cp\u003e1.117\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.72852233676976%\" valign=\"top\"\u003e\n \u003cp\u003e.288\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"7.903780068728523%\" valign=\"top\"\u003e\n \u003cp\u003e.591\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.711340206185568%\" valign=\"top\"\u003e\n \u003cp\u003e1.822\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"21.305841924398624%\" valign=\"top\"\u003e\n \u003cp\u003eAAI LO 2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.807560137457045%\" valign=\"top\"\u003e\n \u003cp\u003e2.418\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.542955326460481%\" valign=\"top\"\u003e\n \u003cp\u003e1.279\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.72852233676976%\" valign=\"top\"\u003e\n \u003cp\u003e3.574\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"7.903780068728523%\" valign=\"top\"\u003e\n \u003cp\u003e.059\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.711340206185568%\" valign=\"top\"\u003e\n \u003cp\u003e11.219\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"21.305841924398624%\" valign=\"top\"\u003e\n \u003cp\u003eAAI LO 3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.807560137457045%\" valign=\"top\"\u003e\n \u003cp\u003e.167\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.542955326460481%\" valign=\"top\"\u003e\n \u003cp\u003e1.306\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.72852233676976%\" valign=\"top\"\u003e\n \u003cp\u003e.016\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"7.903780068728523%\" valign=\"top\"\u003e\n \u003cp\u003e.898\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.711340206185568%\" valign=\"top\"\u003e\n \u003cp\u003e1.182\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"21.305841924398624%\" valign=\"top\"\u003e\n \u003cp\u003eACIEd** LO 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.807560137457045%\" valign=\"top\"\u003e\n \u003cp\u003e1.346\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.542955326460481%\" valign=\"top\"\u003e\n \u003cp\u003e1.186\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.72852233676976%\" valign=\"top\"\u003e\n \u003cp\u003e1.288\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"7.903780068728523%\" valign=\"top\"\u003e\n \u003cp\u003e.256\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.711340206185568%\" valign=\"top\"\u003e\n \u003cp\u003e3.841\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"21.305841924398624%\" valign=\"top\"\u003e\n \u003cp\u003eACIEd LO 2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.807560137457045%\" valign=\"top\"\u003e\n \u003cp\u003e-2.193\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.542955326460481%\" valign=\"top\"\u003e\n \u003cp\u003e1.546\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.72852233676976%\" valign=\"top\"\u003e\n \u003cp\u003e2.012\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"7.903780068728523%\" valign=\"top\"\u003e\n \u003cp\u003e.156\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.711340206185568%\" valign=\"top\"\u003e\n \u003cp\u003e.112\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"21.305841924398624%\" valign=\"top\"\u003e\n \u003cp\u003eACIEd LO 3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.807560137457045%\" valign=\"top\"\u003e\n \u003cp\u003e-1.955\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12.542955326460481%\" valign=\"top\"\u003e\n \u003cp\u003e1.220\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.72852233676976%\" valign=\"top\"\u003e\n \u003cp\u003e2.568\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"7.903780068728523%\" valign=\"top\"\u003e\n \u003cp\u003e.109\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"23.711340206185568%\" valign=\"top\"\u003e\n \u003cp\u003e.142\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e*AAI \u0026ndash; Ambulance Assessment Instrument\u003c/p\u003e\n\u003cp\u003e**ACIEd \u0026ndash; Assessment of Clinical Education\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 3 \u003cem\u003eE\u003c/em\u003e\u003cem\u003effect of the dependent variable\u003c/em\u003e \u0026lsquo;\u003cem\u003eGender\u0026rsquo; on grading related to learning objectives using the Standardized patient for both the AAI and ACIEd with 1 degree of freedom.\u003c/em\u003e\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"552\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.253164556962027%\" valign=\"top\"\u003e\n \u003cp\u003eIndependent variable LOs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.743218806509946%\" valign=\"top\"\u003e\n \u003cp\u003eBeta coefficient\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003eStandard error\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003eWald chi-square test\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.913200723327305%\" valign=\"top\"\u003e\n \u003cp\u003eP-Value\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.540687160940326%\" valign=\"top\"\u003e\n \u003cp\u003eExponentiation of the B coefficient\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.253164556962027%\" valign=\"top\"\u003e\n \u003cp\u003eAAI* LO 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.743218806509946%\" valign=\"top\"\u003e\n \u003cp\u003e.499\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003e1.021\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003e.239\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.913200723327305%\" valign=\"top\"\u003e\n \u003cp\u003e.625\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.540687160940326%\" valign=\"top\"\u003e\n \u003cp\u003e1.647\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.253164556962027%\" valign=\"top\"\u003e\n \u003cp\u003eAAI LO 2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.743218806509946%\" valign=\"top\"\u003e\n \u003cp\u003e.561\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003e.597\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003e.883\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.913200723327305%\" valign=\"top\"\u003e\n \u003cp\u003e.347\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.540687160940326%\" valign=\"top\"\u003e\n \u003cp\u003e1.752\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.253164556962027%\" valign=\"top\"\u003e\n \u003cp\u003eAAI LO 3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.743218806509946%\" valign=\"top\"\u003e\n \u003cp\u003e.540\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003e.825\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003e.428\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.913200723327305%\" valign=\"top\"\u003e\n \u003cp\u003e.513\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.540687160940326%\" valign=\"top\"\u003e\n \u003cp\u003e1.716\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.253164556962027%\" valign=\"top\"\u003e\n \u003cp\u003eACIEd** LO 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.743218806509946%\" valign=\"top\"\u003e\n \u003cp\u003e-.992\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003e.987\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003e1.010\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.913200723327305%\" valign=\"top\"\u003e\n \u003cp\u003e.315\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.540687160940326%\" valign=\"top\"\u003e\n \u003cp\u003e.371\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.253164556962027%\" valign=\"top\"\u003e\n \u003cp\u003eACIEd LO 2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.743218806509946%\" valign=\"top\"\u003e\n \u003cp\u003e.328\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003e.725\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003e.205\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.913200723327305%\" valign=\"top\"\u003e\n \u003cp\u003e.651\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.540687160940326%\" valign=\"top\"\u003e\n \u003cp\u003e1.388\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.253164556962027%\" valign=\"top\"\u003e\n \u003cp\u003eACIEd LO 3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.743218806509946%\" valign=\"top\"\u003e\n \u003cp\u003e-.329\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003e1.092\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.2748643761302%\" valign=\"top\"\u003e\n \u003cp\u003e.091\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.913200723327305%\" valign=\"top\"\u003e\n \u003cp\u003e.763\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.540687160940326%\" valign=\"top\"\u003e\n \u003cp\u003e.720\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e*AAI \u0026ndash; Ambulance Assessment Instrument\u003c/p\u003e\n\u003cp\u003e**ACIEd \u0026ndash; Assessment of Clinical Education\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 4 \u003cem\u003eE\u003c/em\u003e\u003cem\u003effect of the dependent variable\u003c/em\u003e \u0026lsquo;\u003cem\u003eExperience\u0026rsquo; on grading related to Learning objectives using the Standardized patient for both the AAI and ACIEd with 1 degree of freedom.\u003c/em\u003e\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"552\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.508166969147005%\" valign=\"top\"\u003e\n \u003cp\u003eIndependent variable LOs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.793103448275861%\" valign=\"top\"\u003e\n \u003cp\u003eBeta coefficient\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003eStandard error\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003eWald chi-square test\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.1524500907441%\" valign=\"top\"\u003e\n \u003cp\u003eP-Value\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003eExponentiation of the B coefficient\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.508166969147005%\" valign=\"top\"\u003e\n \u003cp\u003eAAI* LO 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.793103448275861%\" valign=\"top\"\u003e\n \u003cp\u003e-.405\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e.970\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e.174\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.1524500907441%\" valign=\"top\"\u003e\n \u003cp\u003e.676\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e.667\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.508166969147005%\" valign=\"top\"\u003e\n \u003cp\u003eAAI LO 2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.793103448275861%\" valign=\"top\"\u003e\n \u003cp\u003e.662\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e.562\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e1.388\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.1524500907441%\" valign=\"top\"\u003e\n \u003cp\u003e.239\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e1.939\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.508166969147005%\" valign=\"top\"\u003e\n \u003cp\u003eAAI LO 3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.793103448275861%\" valign=\"top\"\u003e\n \u003cp\u003e-.527\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e.788\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e.447\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.1524500907441%\" valign=\"top\"\u003e\n \u003cp\u003e.504\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e.590\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.508166969147005%\" valign=\"top\"\u003e\n \u003cp\u003eACIEd** LO 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.793103448275861%\" valign=\"top\"\u003e\n \u003cp\u003e-.204\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e.972\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e.044\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.1524500907441%\" valign=\"top\"\u003e\n \u003cp\u003e.834\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e.815\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.508166969147005%\" valign=\"top\"\u003e\n \u003cp\u003eACIEd LO 2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.793103448275861%\" valign=\"top\"\u003e\n \u003cp\u003e1.918\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e.834\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e5.289\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.1524500907441%\" valign=\"top\"\u003e\n \u003cp\u003e.021*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e6.805\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20.508166969147005%\" valign=\"top\"\u003e\n \u003cp\u003eACIEd LO 3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"13.793103448275861%\" valign=\"top\"\u003e\n \u003cp\u003e-1.260\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e1.160\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e1.181\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.1524500907441%\" valign=\"top\"\u003e\n \u003cp\u003e.277\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16.515426497277677%\" valign=\"top\"\u003e\n \u003cp\u003e.284\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e*AAI \u0026ndash; Ambulance Assessment Instrument\u003c/p\u003e\n\u003cp\u003e**ACIEd \u0026ndash; Assessment of Clinical Education\u0026nbsp;\u003c/p\u003e\n\u003ch2\u003e4.1 Qualitative Results\u003c/h2\u003e\n\u003cp\u003eThe thematic analysis used to explore clinical supervisors\u0026rsquo; experience of assessment in relation to the two different assessment instruments resulted in three themes: \u003cem\u003eLearning objectives\u003c/em\u003e, \u003cem\u003eAssessment\u003c/em\u003e and \u003cem\u003eSupervisorship.\u0026nbsp;\u003c/em\u003e\u003c/p\u003e\n\u003ch2\u003e4.1.1 Learning objectives\u0026nbsp;\u003c/h2\u003e\n\u003cp\u003eSupervisors described a variety of experiences relating to the LO`s in the two sub-themes: Phrasing of the LOs, subjectivity in the LOs, Complexity of the LO`s. The subthemes describe the supervisors\u0026rsquo; view of the learning objectives in relation to how the LO`s are constructed and how they are interpreted as well as how they are used in the daily activities.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePhrasing of the learning objectives:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe supervisors discussed the language used in the ACIEd to describe the LO`s and that the academic writing created confusion and was complex and hard to understand. They also discussed the lack of clear definitions, and difficulties using the ACIEd with several sub-criteria embedded in one LO, which resulted in individual interpretation of the LO and a risk of subjective assessment. According to the supervisors\u0026rsquo; reasoning, several sub-criteria in the same LO complicated the assessment when students performed well according to some of the sub-criteria and poorly according to others. The supervisors expressed that the LOs used in AAI had a clearer phrasing making it easier to use. LO`s related to communication and patient relations were considered complicated to assess using the ACIEd in contrast to LO`s regarding\u0026nbsp;medical procedures where right and wrong was clearly defined. Medical procedures were considered easier to assess in both instruments.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eComplexity of the learning objective\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWhen assessing the students\u0026rsquo; nursing and care skills in the simulations, the supervisors claimed that the complexity of ACIEd made it challenging to explaining why they assessed as they did. In the clinic setting the supervisors said that the complexity of the LO`s made the usage of ACIEd time-consuming and poorly adapted to everyday work, which resulted in limited use, and only used in a summative way right before mid and end assessments. Altogether, the ACIEd was considered by the supervisors to be time-consuming, and leading to subjective interpretations and sometimes, conflicting assessments. The AAI was considered more\u0026nbsp;relevant due to the \u0026ldquo;simplified\u0026rdquo; LO`s. However, AAI\u0026nbsp;was thought to be difficult to use due to the Likert scale ranging from 1-7 were lack of clear definitions of the grading steps increased the complexity of assessing the LO`s.\u003c/p\u003e\n\u003ch2\u003e4.1.2 Assessment\u003c/h2\u003e\n\u003cp\u003eSupervisors described assessment from different perspectives as presented below in the following sub-themes: Summative Vs Formative assessment,\u0026nbsp;Subjectivity in assessment,\u0026nbsp;Pass/Fail or Likert scale, and\u0026nbsp;Supervisors experience related to assessments. Within the sub-themes there were variations in how the supervisors described their experiences.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSummative Vs Formative assessment\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe supervisors discussed if and how the assessment instruments could be used for formative or summative assessment and concluded that both instruments could be used for both purposes. However, the supervisors discussed whether ACIEd was harder to use as a formative instrument due to the sub-criteria in the LO`s and that the pass/fail scale were less useful then the 7 graded Likert scale when it came to displaying progress. The supervisors discussed whether displaying the student\u0026rsquo;s progression had a positive pedagogical value and they concluded as preferable relating it to formative feedback. Supervisors also discussed the frequency of formative assessments. Some supervisors raised concerns related to the increased workload generated by daily documented, formative feedback.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSubjectivity in assessment\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo decrease the subjectivity, the supervisors discussed involving the patients in the assessments of students\u0026rsquo; performance. They argued that the only persons who could assess the LOs concerning the patients\u0026rsquo; own perception were the patients themselves.\u0026nbsp;The supervisors concluded that more care encounters, with assessments between every encounter would probably produce a more accurate assessment of the student\u0026rsquo;s nursing skills. The supervisors also discussed the risk that the students would only be assessed according to the supervisor\u0026rsquo;s interpretation of what is a pass performance in relation to the assessment instrument.\u003c/p\u003e\n\u003cp\u003eThe supervisors found that the assessments with a Likert scale could be beneficial but that the grading steps needed to be carefully defined to avoid subjective assessments. The definitions could be made clearer by using examples in relations to the grading steps and the LO`s where requirements could be listed for each step.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eSupervisors said that assessments using a Pass/Fail grading was too definite. Supervisors described that assessing the care encounters became complicated since student\u0026rsquo;s performances may contain good and bad performances and with Pass/Fail grading the nuances did not become clear.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSupervisors experience related to assessments.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eSupervisors\u0026rsquo; description of their relationship to the LO`s varied where some had more issues than others. Supervisors with less experience were more critical towards the LO`s then experienced supervisors. Furthermore, supervisors with less experience discussed that the student were obligated to display skills and knowledge and that supervisors were obligated to assess in accordance with the LO`s, meaning responsibility for demonstrating knowledge and skills fell on the student.\u003c/p\u003e\n\u003cp\u003eThe more experienced supervisors argued that the LO`s were more like guidelines than specific goals to achieve, and that they used LO`s as a basis for discussions with the students. They argued that their clinical experience and understanding were the basis for the assessment, making the challenge with the LO`s less important, which contrasted with the view of the less experienced supervisors who interpreted the LO\u0026acute;s more literally. Furthermore, the more experienced supervisors discussed that complex care encounters offered scarce opportunities for the students to display knowledge and skills and that several care encounters were needed to assess knowledge and skills over time. \u0026nbsp;\u003c/p\u003e\n\u003ch2\u003e4.1.3 Supervisorship\u003c/h2\u003e\n\u003cp\u003eThe supervisorship was described by the supervisors in the following two sub-themes: Relationship with the students and Teams of supervisors.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u0026nbsp;Relationship with the students\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll supervisors said that a relationship with the student could be a confounder when assessing students. They discussed whether a good relationship with the student would probably result in a more favourable assessment. The supervisors also claimed that a troubled relationship with the student might result in disinterest from their perspective which could result in diminishing feedback and lowered clinical education quality resulting in higher risk for failure. The supervisors discussed that formative feedback could help detect a lack of progression at an early stage and that measures could be taken to improve the situation for the student. Among the female supervisors a recurring statement was that it was difficult to assess the student negatively in the simulations because it felt harsh or even cruel to fail a student. Among the male supervisor\u0026rsquo;s similar feelings were described, but they related to the fact that assessments are not carried out on single occasions but over time, making the assessment instrument unfit for this kind of assessment. Male supervisors argued to a greater extent that it was hard to assess the student in the simulation due to lack of information about both patients and students. They argued that although some simulations were not as described in \u0026ldquo;textbook\u0026rdquo; examples, guidelines were bent daily and therefore it would be unfair to expect \u0026ldquo;textbook\u0026rdquo; care from the students. Hence, the male supervisors argued that the assessment instrument was a tool used for discussion and could not stand alone which it did in the simulations.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTeams of supervisors\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe supervisors highlighted continuity as something important for the student\u0026rsquo;s learning, but it could also complicate their assessment since they may develop a relationship with the student. \u0026nbsp;The desire for continuity in supervisorship could also create a problem when working schedules changed. This could result in the involvement of other supervisors, and disruption in the individual learning plan. The supervisors with longer experienced argued that it could be favourable with more than one supervisor involved in the assessment of the student, due to different perspectives. They argued that a supervisor team with different combinations of knowledge could be beneficial for the student, but that such teams must be coordinated and documented to ensure that the student\u0026rsquo;s learning progress was not hindered.\u0026nbsp;\u003c/p\u003e"},{"header":"5. Discussion","content":"\u003cp\u003eThe findings showed that there were no significant differences in pass/fail gradings using the two different assessment instruments with the same content in the LOs, meaning that no matter what instrument was used the grade was the same when supervisors assessed the students in the simulated scenarios. However, there were significant differences within the assessment instruments, but the differences were consistent between the instruments, meaning that the quality of the assessment was considered equal. The differences within the assessment instrument can be explained by supervisor bias. Chong \u003cem\u003eet al\u003c/em\u003e showed in their study that seniority was a source for bias in LO`s related to communication but that the bias did not persist in LO`s related to physical examinations which is in line with the findings in this study [8]. The interviews with the clinical supervisors provided a more vivid picture of the complexity of assessments. Firstly, the intention of the assessments must be made clear for everyone. If the objective is formative assessment, the data indicates that using Likert scales is preferable to display progression. The complexity, phrasing and definition were recurring in the interviews, and it is a worth discussing why this is an issue. Supervisors rarely had any training in how to use the assessment instruments and as described, the more experience supervisors did not excel in their knowledge of the instruments but used their clinical experience to assess students from their own perspective. With this logic it would be wise to invest in training the supervisors rather than simplifying the LO\u0026acute;s. Prior research has shown that support from the supervisors during the clinical training is crucial to create a positive learning environment and to improve assessments [31].\u003c/p\u003e \u003cp\u003eHowever, the need for supervisor training and support does not mean that improving the assessment instrument is unnecessary. If a Likert scale is used, a clear definition of the grading steps is important as well as the layout of the instrument. As described by Immonen et al, it is important that the instrument is adapted to the every-day work, and fast and easy to use without losing its reliability and validity [14].\u003c/p\u003e \u003cp\u003eSubjectivity and complexity were reoccurring statements during the interviews. The supervisors highlighted this in all aspects regarding the clinical education. The supervisors\u0026rsquo; own experience\u0026rsquo; and expectations play a vital part in the clinical education as well as the relationship between the supervisor and the students which is supported in prior research [24]. To decrease subjectivity, an increase in the number of assessments made by more than one supervisor could be beneficial. Assessments after every care encounter in the ambulance service would generate a good basis for an overall assessment of the students\u0026rsquo; performance. By organising the supervisors in teams, the different knowledges of the supervisors could be effectively utilized and possibly decrease subjectivity resulting in an improvement of the validity of the assessment. In contrast to this, prior research has shown that the relationship between the student and the supervisor is important to build trust and thereby a positive learning environment [30]. With documented, formative assessments accessible for both students and supervisors, continuity could be created through communication between supervisors. By using digital devices with LO`s and a Likert scale prepared in a mobile application, the assessments could be made easier to access, faster to use and the results could be displayed as a progression curve visible for students and supervisors. Digitalization of the assessment instrument also holds advantages concerning student possibility to argue for their grades. Without documented progress the students are in the hands of the supervisors as the only source of information about their performances. To further strengthen the validity of the assessments, other sources of feedback could be used. The supervisors discussed involving patients in the assessments and concluded that this could reduce subjectivity in the assessments and add other perspectives to the assessment. Further research is needed to fully understand the complexity of assessments and what methods to use to improve the quality of the assessment and strengthen the students learning.\u003c/p\u003e \u003cdiv id=\"Sec16\" class=\"Section2\"\u003e \u003ch2\u003e5.1 Methodological considerations\u003c/h2\u003e \u003cp\u003eThere are several limitations to this study. The number of supervisors included in the study was decided in discussion with the research team and a statistician after considering the availability of clinical supervisors and the extent of data needed for analysis. No power calculations were performed. Using standardised patients and simulation for assessing and grading students\u0026rsquo; ability to care for patients may not reflect the clinical reality. However, since every care encounter is unique it would have been difficult to conduct a similar study in a clinical setting. Another limitation concerns the qualitative data collection. The interviewer is well known in the ambulance service, and this may have affected how the participants\u0026rsquo; discussions concerning their experiences of using the assessment instruments in both a positive and negative way and increase the risk of bias. Conducting interviews between ambulance missions may also be considered as a limitation since the time for interviews was limited which could have resulted in participants not being able to develop their reasoning. The interviews were documented through field notes, and important information could have been missed. Field notes limit the possibility of quotations, which could have strengthened the validity of the findings in the interviews.\u003c/p\u003e \u003c/div\u003e"},{"header":"6. Conclusion","content":"\u003cp\u003eThe findings showed that there were no significant differences in pass/fail gradings using the two different assessment instruments containing the same LOs. However, the qualitative data suggests that supervisors struggled with subjectivity in the assessments as well as phrasing and definitions of the LO\u0026acute;s and the scales used in both instruments. This resulted in arbitrary assessments that were time-consuming and resulted in limited usage in the day-to-day assessment. The supervisors argued that the AAI was better adapted for formative assessment due to its Likert scale and simplified LOs, but a clear definition of the grading scales was considered important. Further research is needed concerning the validity of the assessments and how teams of supervisors can utilize different perspectives to improve the quality of the assessments. Digitalization could play a vital role in documenting feedback from multiple sources to enhance the formative feedback given to students during their clinical training. More research is needed to fully understand the mechanism behind the subjectivity of assessments and what methods could be used to strengthen the quality of the assessments and improve the quality of the clinical education.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cp\u003eAAI Ambulance Assessment Instrument \u003c/p\u003e\n\u003cp\u003eACIEd Assessment of Clinical Education \u003c/p\u003e\n\u003cp\u003eLO Learning objectives \u003c/p\u003e\n\u003cp\u003eOSCE Observed Structural Clinical Examination\u003c/p\u003e\n\u003cp\u003eRN Registered nurses \u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to practice\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA written, informed consent was sign by all participants before participation of the study.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe study was designed according to the ethical principles for research described by the International Council of Nurses [13]. The study was approved by the\u0026nbsp;Swedish Ethical Review Authority, Stockholm, Sweden (2016/594-31).\u0026nbsp;All methods were performed in accordance with the relevant guidelines and regulations stipulated in the Helsinkideclaration.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent to publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNo individual data or other sources of data requiring consent for publication were used in this article.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe dataset generated and analysed in this study is available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors reported no competing financial or non-financial interests. All authors contributed significantly to the content of this article.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFunding for this study was provided by the Karolinska Institute.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll authors contributed significantly to the study design in terms of the choice of analytic methods and interview design.\u003c/p\u003e\n\u003cp\u003eAll authors played an active role in reading and providing feedback about the material throughout the entire process.\u003c/p\u003e\n\u003cp\u003eTN, EB and VL were involved in carrying out group interviews of the students and contributed significantly by reading and providing feedback about the manuscript.\u003c/p\u003e\n\u003cp\u003eAll authors reviewed and approved the submitted manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eI\u0026rsquo;ve would like to send a special thanks to my family for their never wavering support and to all participants how made this paper possible. I also wanted to extend my gratitude\u0026rsquo;s towards Ambulansen I StorStockholm AB for facilitating my studies and this paper.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eArrogante, O., Gonz\u0026aacute;lez-Romero, G. M., L\u0026oacute;pez-Torre, E. M., Carri\u0026oacute;n-Garc\u0026iacute;a, L., \u0026amp; Polo, A. (2021). Comparing formative and summative simulation-based assessment in undergraduate nursing students: nursing competency acquisition and clinical simulation satisfaction. \u003cem\u003eBMC Nurs, 20\u003c/em\u003e(1), 92. doi:10.1186/s12912-021-00614-2\u003c/li\u003e\n\u003cli\u003eBenner, P. (1982). From novice to expert. \u003cem\u003eAm J Nurs, 82\u003c/em\u003e(3), 402-407. \u003c/li\u003e\n\u003cli\u003eBerkhout, J. J., Helmich, E., Teunissen, P. W., van den Berg, J. W., van der Vleuten, C. P., \u0026amp; Jaarsma, A. D. (2015). Exploring the factors influencing clinical students\u0026apos; self-regulated learning. \u003cem\u003eMed Educ, 49\u003c/em\u003e(6), 589-600. doi:10.1111/medu.12671\u003c/li\u003e\n\u003cli\u003eBerkhout, J. J., Helmich, E., Teunissen, P. W., van der Vleuten, C. P. M., \u0026amp; Jaarsma, A. D. C. (2018). Context matters when striving to promote active and lifelong learning in medical education. \u003cem\u003eMed Educ, 52\u003c/em\u003e(1), 34-44. doi:10.1111/medu.13463\u003c/li\u003e\n\u003cli\u003eBraun, V., \u0026amp; Clarke, V. (2006). Using thematic analysis in psychology. \u003cem\u003eQualitative Research in Psychology, 3\u003c/em\u003e(2), 77-101. doi:10.1191/1478088706qp063oa\u003c/li\u003e\n\u003cli\u003eBrown, J. M., Lowe, K., Fillingham, J., Murphy, P. N., Bamforth, M., \u0026amp; Shaw, N. J. (2014). An investigation into the use of multi-source feedback (MSF) as a work-based assessment tool. \u003cem\u003eMed Teach, 36\u003c/em\u003e(11), 997-1004. doi:10.3109/0142159x.2014.909920\u003c/li\u003e\n\u003cli\u003eBurke, E., Kelly, M., Byrne, E., Ui Chiardha, T., Mc Nicholas, M., \u0026amp; Montgomery, A. (2016). Preceptors\u0026apos; experiences of using a competence assessment tool to assess undergraduate nursing students. \u003cem\u003eNurse Educ Pract, 17\u003c/em\u003e, 8-14. doi:10.1016/j.nepr.2016.01.004\u003c/li\u003e\n\u003cli\u003eChong, L., Taylor, S., Haywood, M., Adelstein, B. A., \u0026amp; Shulruf, B. (2018). Examiner seniority and experience are associated with bias when scoring communication, but not examination, skills in objective structured clinical examinations in Australia. \u003cem\u003eJ Educ Eval Health Prof, 15\u003c/em\u003e, 17. doi:10.3352/jeehp.2018.15.17\u003c/li\u003e\n\u003cli\u003eCoburn, C. V., Gilland, D., Owen, M., \u0026amp; Amar, A. (2018). Ambulatory care education: Preparing nurses for the future of healthcare. \u003cem\u003eNurse Educ Today, 66\u003c/em\u003e, 79-81. doi:10.1016/j.nedt.2018.03.015\u003c/li\u003e\n\u003cli\u003eFluit, C., Bolhuis, S., Grol, R., Ham, M., Feskens, R., Laan, R., \u0026amp; Wensing, M. (2012). Evaluation and feedback for effective clinical teaching in postgraduate medical education: validation of an assessment instrument incorporating the CanMEDS roles. \u003cem\u003eMed Teach, 34\u003c/em\u003e(11), 893-901. doi:10.3109/0142159x.2012.699114\u003c/li\u003e\n\u003cli\u003eFranklin, N., \u0026amp; Melville, P. (2015). Competency assessment tools: An exploration of the pedagogical issues facing competency assessment for nurses in the clinical environment. \u003cem\u003eCollegian, 22\u003c/em\u003e(1), 25-31.\u003c/li\u003e\n\u003cli\u003eGibbons, S. W., Adamo, G., Padden, D., Ricciardi, R., Graziano, M., Levine, E., \u0026amp; Hawkins, R. (2002). Clinical evaluation in advanced practice nursing education: using standardized patients in Health Assessment. \u003cem\u003eJ Nurs Educ, 41\u003c/em\u003e(5), 215-221.\u003c/li\u003e\n\u003cli\u003eICN. (2012). The ICN Code of ethics for nurses.\u003c/li\u003e\n\u003cli\u003eImmonen, K., Oikarainen, A., Tomietto, M., K\u0026auml;\u0026auml;ri\u0026auml;inen, M., Tuomikoski, A. M., Kaučič, B. M., . . . Mikkonen, K. (2019). Assessment of nursing students\u0026apos; competence in clinical practice: A systematic review of reviews. \u003cem\u003eInt J Nurs Stud, 100\u003c/em\u003e, 103414. doi:10.1016/j.ijnurstu.2019.103414\u003c/li\u003e\n\u003cli\u003eKibble, J. D. (2017). Best practices in summative assessment. \u003cem\u003eAdv Physiol Educ, 41\u003c/em\u003e(1), 110-119. doi:10.1152/advan.00116.2016\u003c/li\u003e\n\u003cli\u003eLindstr\u0026ouml;m, V., Bohm, K., \u0026amp; Kurland, L. (2015). Prehospital care in Sweden. \u003cem\u003eNotfall + Rettungsmedizin, 18\u003c/em\u003e(2), 107-109. doi:10.1007/s10049-015-1989-1\u003c/li\u003e\n\u003cli\u003eMcCarthy, B., \u0026amp; Murphy, S. (2008). Assessing undergraduate nursing students in clinical practice: do preceptors use assessment strategies? \u003cem\u003eNurse Educ Today, 28\u003c/em\u003e(3), 301-313. doi:10.1016/j.nedt.2007.06.002\u003c/li\u003e\n\u003cli\u003eMelby, V. (2000). Experiential learning in pre-hospital emergency care: a qualitative study. \u003cem\u003eNurse Educ Today, 20\u003c/em\u003e(8), 638-645. doi:10.1054/nedt.2000.0496\u003c/li\u003e\n\u003cli\u003eNilsson, T., \u0026amp; Lindstrom, V. (2016). Clinical decision-making described by Swedish prehospital emergency care nurse students - An exploratory study. \u003cem\u003eInt Emerg Nurs, 27\u003c/em\u003e, 46-50. doi:10.1016/j.ienj.2015.10.006\u003c/li\u003e\n\u003cli\u003eNilsson, T., \u0026amp; Lindstrom, V. (2017). Nursing students\u0026apos; perceptions of learning nursing skills in the ambulance service. \u003cem\u003eNurse Educ Pract, 24\u003c/em\u003e, 1-5. doi:10.1016/j.nepr.2017.02.011\u003c/li\u003e\n\u003cli\u003eNyqvist, J., Brolin, K., Nilsson, T., \u0026amp; Lindstr\u0026ouml;m, V. (2020). The learning environment and supportive supervision promote learning and are based on the relationship between students and supervisors - A qualitative study. \u003cem\u003eNurse Educ Pract, 42\u003c/em\u003e, 102692. doi:10.1016/j.nepr.2019.102692\u003c/li\u003e\n\u003cli\u003ePatr\u0026iacute;cio, M. F., Juli\u0026atilde;o, M., Fareleira, F., \u0026amp; Carneiro, A. V. (2013). Is the OSCE a feasible tool to assess competencies in undergraduate medical education? \u003cem\u003eMed Teach, 35\u003c/em\u003e(6), 503-514. doi:10.3109/0142159x.2013.774330\u003c/li\u003e\n\u003cli\u003eSchoonenboom, J., \u0026amp; Johnson, R. B. (2017). How to Construct a Mixed Methods Research Design. \u003cem\u003eKolner Z Soz Sozpsychol, 69\u003c/em\u003e(Suppl 2), 107-131. doi:10.1007/s11577-017-0454-1\u003c/li\u003e\n\u003cli\u003eSundstrom, B. W., \u0026amp; Dahlberg, K. (2012). Being prepared for the unprepared: a phenomenology field study of Swedish prehospital care. \u003cem\u003eJ Emerg Nurs, 38\u003c/em\u003e(6), 571-577. doi:10.1016/j.jen.2011.09.003\u003c/li\u003e\n\u003cli\u003eTen Cate, O., \u0026amp; Regehr, G. (2019). The Power of Subjectivity in the Assessment of Medical Trainees. \u003cem\u003eAcad Med, 94\u003c/em\u003e(3), 333-337. doi:10.1097/acm.0000000000002495\u003c/li\u003e\n\u003cli\u003eUlfvarson, J., \u0026amp; Oxelmark, L. (2012). Developing an assessment tool for intended learning outcomes in clinical practice for nursing students. \u003cem\u003eNurse Education Today, 32\u003c/em\u003e(6), 703-708. doi:https://doi.org/10.1016/j.nedt.2011.09.010\u003c/li\u003e\n\u003cli\u003eUlfvarson, J., Oxelmark, L., \u0026amp; Jirwe, M. (2018). Assessment in clinical education: A comparison between a generic instrument and a course-specific criterion-based instrument. \u003cem\u003eNordic Journal of Nursing Research\u003c/em\u003e, 205715851876840. doi:10.1177/2057158518768404\u003c/li\u003e\n\u003cli\u003evan de Ridder, J. M., Stokking, K. M., McGaghie, W. C., \u0026amp; ten Cate, O. T. (2008). What is feedback in clinical education? \u003cem\u003eMed Educ, 42\u003c/em\u003e(2), 189-197. doi:10.1111/j.1365-2923.2007.02973.x\u003c/li\u003e\n\u003cli\u003eVan Horn, E., \u0026amp; Christman, J. (2017). Assessment of Nursing Student Confidence Using the Clinical Skills Self-Efficacy Scale. \u003cem\u003eNurs Educ Perspect, 38\u003c/em\u003e(6), 344-346. doi:10.1097/01.Nep.0000000000000169\u003c/li\u003e\n\u003cli\u003eWallin, K., Fridlund, B., \u0026amp; Thor\u0026eacute;n, A. B. (2013). Prehospital Emergency Nursing students\u0026apos; experiences of learning during prehospital clinical placements. \u003cem\u003eInt Emerg Nurs, 21\u003c/em\u003e(3), 197-203. doi:10.1016/j.ienj.2012.09.003\u003c/li\u003e\n\u003cli\u003eWallin, K., H\u0026ouml;rberg, U., Harst\u0026auml;de, C. W., Elmqvist, C., \u0026amp; Bremer, A. (2020). Preceptors\u0026acute; experiences of student supervision in the emergency medical services: A qualitative interview study. \u003cem\u003eNurse Educ Today, 84\u003c/em\u003e, 104223. doi:10.1016/j.nedt.2019.104223\u003c/li\u003e\n\u003cli\u003eWireklint Sundstrom, B., Bremer, A., Lindstrom, V., \u0026amp; Vicente, V. (2018). Caring science research in the ambulance services: an integrative systematic review. \u003cem\u003eScand J Caring Sci\u003c/em\u003e. doi:10.1111/scs.12607\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-3920417/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3920417/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground: \u003c/strong\u003eAssessment of undergraduate students using assessment instruments in the clinical setting is known to be complex. The aim of this study was therefore to examine whether two different assessment instruments, containing learning objectives (LO`s) with similar content, results in similar assessments by the clinical supervisors and to explore clinical supervisors’ experiences of assessment regarding the two different assessment instruments.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethod:\u003c/strong\u003e A mixed-methods approach was used. Four simulated care encounter scenarios were evaluated by 50 supervisors using two different assessment instruments. 28 follow-up interviews were conducted. Descriptive statistics and logistic binary regression were used for quantitative data analysis, along with qualitative thematic analysis of interview data.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResult:\u003c/strong\u003e While significant differences were observed within the assessment instruments, the differences were consistent between the two instruments, indicating that the quality of the assessment instruments were considered equivalent. Supervisors noted that the relationship between the students and supervisors could introduce subjectivity in the assessments and that working in groups of supervisors could be advantageous. In terms of formative assessments, the Likert scale was considered a useful tool for evaluating learning objectives. However, supervisors had different views on grading scales and the need for clear definitions. The supervisors concluded that a complicated assessment instrument led to limited very-day usage and did not facilitate formative feedback. Furthermore, supervisors discussed how their experiences influenced the use of the assessment instruments, which resulted in different descriptions of the experience. These differences led to a discussion of the need of supervisor teams to enhance the validity of assessments.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusion:\u003c/strong\u003e The findings showed that there were no significant differences in pass/fail gradings using the two different assessment instruments. The quantitative data suggests that supervisors struggled with subjectivity, phrasing, and definitions of the LO´s and the scales used in both instruments. This resulted in arbitrary assessments that were time-consuming and resulted in limited usage in the day-to-day assessment. To mitigate the subjectivity, supervisors suggested working in teams and conducting multiple assessments over time to increase assessment validity.\u003c/p\u003e","manuscriptTitle":"Assessment during clinical education among nursing students using two different assessment instruments.","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-02-23 04:22:35","doi":"10.21203/rs.3.rs-3920417/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-05-25T00:38:40+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-05-24T14:31:33+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"268237503812632286735821099365585730427","date":"2024-05-09T01:52:07+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"f2ffe912-43dd-4b44-9cb2-f3eb104fe5e9","date":"2024-03-24T05:31:55+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-03-05T11:42:07+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"4ed8475a-4fb6-478e-a3c1-f3e9f704c9c9","date":"2024-02-25T05:46:44+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-02-25T01:08:19+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-02-24T22:30:34+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-02-21T17:13:18+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-02-21T17:07:45+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Education","date":"2024-02-02T09:48:04+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"7e713c78-47fe-43a7-80ac-3ba981dd8798","owner":[],"postedDate":"February 23rd, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2024-08-12T16:09:51+00:00","versionOfRecord":{"articleIdentity":"rs-3920417","link":"https://doi.org/10.1186/s12909-024-05771-x","journal":{"identity":"bmc-medical-education","isVorOnly":false,"title":"BMC Medical Education"},"publishedOn":"2024-08-07 15:57:52","publishedOnDateReadable":"August 7th, 2024"},"versionCreatedAt":"2024-02-23 04:22:35","video":"","vorDoi":"10.1186/s12909-024-05771-x","vorDoiUrl":"https://doi.org/10.1186/s12909-024-05771-x","workflowStages":[]},"version":"v1","identity":"rs-3920417","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3920417","identity":"rs-3920417","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}