Optimising the diagnostic accuracy of First post-contrAst SubtracTed breast MRI (FAST MRI) through interpretation-training: a multicentre e-learning study, mapping the learning curve of NHS Breast Screening Programme (NHSBSP) mammogram readers using an enriched dataset

Optimising the diagnostic accuracy of First post-contrAst SubtracTed breast MRI (FAST MRI) through interpretation-training: a multicentre e-learning study, mapping the learning curve of NHS Breast Screening Programme (NHSBSP) mammogram readers using an enriched dataset | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Optimising the diagnostic accuracy of First post-contrAst SubtracTed breast MRI (FAST MRI) through interpretation-training: a multicentre e-learning study, mapping the learning curve of NHS Breast Screening Programme (NHSBSP) mammogram readers using an enriched dataset Lyn I Jones, Andrea Marshall, Rebecca Geach, Premkumar Elangovan, and 8 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-3881738/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 7 You are reading this latest preprint version Abstract Background: Abbreviated breast MRI (FAST MRI) is being introduced into clinical practice. Specificity optimisation is essential to minimise harm through false positive results for populations with low pre-test probability. This study aimed to optimise diagnostic accuracy through the adaptation of a FAST MRI interpretation-training programme. Methods: A FAST MRI interpretation-training programme was adapted to provide additional training during the assessment task (interpretation of an enriched dataset of 125 FAST MRI scans) by giving readers feedback about the true outcome of each scan immediately after each scan was interpreted (formative assessment). The training programme was additionally adapted for remote e-learning delivery. Study design: prospective, blinded interpretation of an enriched dataset by multiple readers. Results: 43 mammogram readers completed the training, 22 who interpreted breast MRI in their clinical role (Group 1) and 21 who did not (Group 2). Overall sensitivity was 83% (95%CI 81-84%; 1994/2408), specificity 94% (95%CI 93–94%; 7806/8338), readers’ agreement with the true outcome kappa = 0.75 (95%CI 0.74-0.77) and diagnostic odds ratio = 70.67 (95%CI 61.59-81.09). Group 1 readers showed similar sensitivity (84%) to Group 2 (82% p=0.14), but slightly higher specificity (94% v. 93%, p=0.001). Concordance with the ground truth increased significantly with the number of FAST MRI scans read through the formative assessment task (p=0.002) but by differing amounts depending on whether or not a reader had previously attended FAST MRI training (interaction p=0.02). Concordance with the ground truth was significantly associated with reading batch size (p=0.02), tending to worsen when more than 50 scans were read per batch. Group 1 took a median of 56 seconds (range 8-47466) to interpret each FAST MRI scan compared with 78 seconds (14-22830, p <0.0001) for Group 2. Conclusions: Provision of immediate feedback to mammogram readers during the assessment test set reading task increased specificity for FAST MRI interpretation and achieved high diagnostic accuracy. Optimal reading-batch size for FAST MRI was 50 reads per batch. Trial registration (25/09/2019) : ISRCTN16624917 FAST MRI abbreviated breast MRI breast cancer screening formative assessment medical education diagnostic accuracy e-learning Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Background Screening with breast MRI can reduce interval cancers for women with very dense breasts but otherwise at population-risk of breast cancer( 1 , 2 ). First post-contrAst SubtracTed MRI (FAST MRI), a shortened form of breast MRI, retains the sensitivity for aggressive breast cancer of full protocol breast MRI (fpMRI)( 3 , 4 ), and its shorter acquisition and interpretation times make it more likely to be cost-effective( 5 ). The diagnostic accuracy of FAST MRI has been shown to be similar to that of fpMRI when reported by experts in fpMRI interpretation( 6 , 7 ). FAST MRI is being introduced into clinical practice to screen a wider group of women than those currently screened with fpMRI( 8 , 9 ). Internationally, many fewer radiologists interpret fpMRI than interpret screening mammograms( 10 ). Published expert opinion on the implementation of FAST MRI into screening practice has emphasized the importance of performance audit for readers whilst suggesting that benchmarks for interpretation can be developed following roll out without specific interpretation-training for existing fpMRI readers( 8 ). However, by excluding mammogram readers who do not currently interpret fpMRI from FAST MRI interpretation, this approach could limit the potential expansion of the role of FAST MRI by limiting the numbers of radiologists who may interpret it. For FAST MRI to be scaled up within screening programmes worldwide, effective FAST MRI interpretation-training for mammogram readers is needed. The Society of Breast MRI provided interpretation-training to experienced fpMRI readers prior to their interpretation of FAST MRI within the EA1141 breast screening trial( 3 ). No formal evaluation of this training was published but diagnostic accuracy achieved at single read within this trial for FAST MRI was 96% sensitivity and 87% specificity( 3 ). We could find only one published multi-centre study evaluating the effectiveness of FAST MRI interpretation-training( 11 ). This study built on previous work to develop a standardised training programme for NHS Breast Screening Programme (NHSBSP) mammogram readers( 12 , 13 ). Following a single day’s training, mammogram readers achieved an overall sensitivity of 86% and specificity of 86%. However, the diagnostic accuracy achieved in the study by those with previous experience of reporting fpMRI (sensitivity 89%, specificity 90%) remained higher than for those with no such previous experience (sensitivity 83% p < 0.001, specificity 82% p < 0.001)( 11 ). Formative assessment is an educational technique where the assessment process includes feedback to the learner so that in addition to measuring the learner’s achievement, it enhances learning( 18 ). We hypothesised that readers’ diagnostic accuracy could be optimised by converting summative assessment (without any feedback to the readers), as used in previous FAST MRI reader training studies( 11 , 13 ) into formative assessment (by giving readers immediate feedback for each FAST MRI scan read during the assessment task). Methods This study was reviewed and approved by the Health and Care Research Wales Ethics Committee and by the Health Research Authority (REC:21/HRA/4543 IRAS:301714) and prospectively registered (ISRCTN:16624917) and all participants gave written informed consent. The aims of the study were: To determine whether mammogram readers’ diagnostic accuracy can be improved through the addition of formative assessment( 18 ) to standardised FAST MRI interpretation-training To map the learning curve for NHSBSP mammogram readers during FAST MRI interpretation-training (by evaluating the incremental diagnostic accuracy of both experienced and novice MRI readers during the formative assessment process). Study Design Prospective, blinded interpretation of an enriched dataset by multiple readers. Participants and setting All NHSBSP mammogram readers who were fully qualified to interpret mammograms, at 7 sites in England were invited to take part (December 2021 – February 2022) and were classified as Group 1 if they also interpreted fpMRI in their normal clinical practice, and Group 2 if not. Both Groups were sub-classified into those who had previously undertaken in-person FAST MRI training as part of a research study (“Attended”) and those that had not (“Not attended”). Participants then independently completed two days of standardised FAST MRI interpretation-training, of which the second day comprised reading a test set of FAST MRI scans with feedback on the true outcome for each scan being given immediately after their opinion was recorded (formative assessment( 18 )). The training was undertaken remotely at times chosen by the readers (January - June 2022). Test set The test set comprised 125 FAST MRI scans with known outcome, acquired as fpMRI during 2015, including a consecutive high-risk screening series (72 scans) enriched with additional cancer cases from the same year (53 scans). All cancer cases had histological confirmation and non-cancer scans were confirmed with two-year follow-up. Details of this test set have been described previously( 11 , 13 ) (FAST MRI specification and test set composition have been reproduced in Additional file 1 ). Of 125 FAST MRIs in the test set, 54 had biopsy-confirmed unilateral cancer and one bilateral (56 breasts with cancer) and 2 women had two separate tumours identified in the same breast, giving a total of 58 cancers reported in the ground truth, 56 invasive and 2 ductal carcinoma-in-situ (DCIS). The current study used the same test set as the previous interpretation-training studies( 11 , 13 ). The training in the current study differed from that delivered previously in being delivered as remote e-learning and including additional formative assessment. Participants who had taken part in a previous interpretation-training study (11 in Group 1 and 7 in Group 2) were viewing the FAST MRI scans of the test set for the second time. However, prior to the start of the current study, the ground truth (true outcome) of the test set had at no time been revealed to them and the average time interval between reading the test set in one of the two previous studies and in the current study was 24 months (range 17–30 months). Electronic format Previously developed software (RiViewer), that displays FAST MRI as maximum intensity projection (MIP) and stacked, subtracted slices, was used in which biopsy-proven cancers had been drawn onto images electronically as regions of interest (ROI volumes) to provide ground truth. As in a previous study( 11 ), during hands-on workstation training, learners reviewed 29 training FAST MRI scans, and could discover the ground truth at the touch of a button. In the current study the software was adapted so that the test set was presented as formative assessment (providing a second, additional day of training). Once participants had completed their interpretation of each scan and committed their assessment electronically, it was automatically locked in and they were immediately able to view the ground truth of the scan, superimposed on their own opinion. This gave them instant feedback prior to their viewing of the next scan in the test set. Training and test set MRIs are mutually exclusive and were from a single centre but acquired during different years, from different women. The readers’ point ROI needed to be within the ground truth ROI volume for their opinion to be registered as correct. Standardised training A previously developed standardised training programme( 11 – 13 ) (details reproduced in Appendix 2 ), was adapted to enable remote self-directed independent e-learning by participants. Previously developed small-group presentations and hands-on workstation sessions were recorded and made available to participants online, as videos. Additionally, software was provided to the NHS sites that enabled learners to simultaneously login to the RiViewer on NHS workstations to practice image manipulation of the 29 training FAST MRI scans, guided by the recorded sessions. Readers were taught how to classify FAST MRI scans according to the UK 5-point breast imaging classification specified for screening fpMRI in women at higher risk of breast cancer within NHSBSP( 19 ). When adding a point region of interest (ROI) to an image they were prompted to label the ROI with an MRI classification from the UK 5-point scale. Quantification of the UK 5-point scale, defining how it maps to the BI-RADS classification system was described by Taylor et al.( 20 ). In the current study, the new provision of immediate feedback (on the true outcome of each scan) during the test set reading assessment task (termed formative assessment( 18 )) forms a new and additional part of the reader training. Test set interpretation Having completed the training set, participants interpreted the test set of 125 FAST MRIs, blinded to all other information (clinical history, previous imaging, histology, and other readers’ interpretations). Readers were told to expect more cancers than in usual screening practice but no other indication of the number of cancers was given. The test set was presented to each reader in a different random order. For the current study, readers were encouraged to complete their reading of the assessment test set (formative assessment task) within as short a time as was reasonably possible, following completion of the other training material. No recommendations were made regarding the number of scans to be read at a time (batch size). Sample size calculation Using the results of a previous interpretation-training study( 13 ), a sample of 250 breasts from 125 women would allow the lower 95% confidence limit of the inter-rater reliability (Kappa statistic) to be estimated to within 0.07 with a minimum of 6 readers in each group and a proportion of cancers of 0.22( 21 ). Thus, to assess inter-rater reliability (we required a minimum total of 12 readers: 6 in each group. Statistical analysis Per-breast analysis of the frequency of results against true outcome was obtained overall and for each reader. Sensitivity, specificity, and concordance of readers’ FAST MRI classification with the true outcome were determined and differences across reader groups and previous attendance on a FAST MRI training session assessed using a multi-level-generalised-mixed model to account for multiple readers per scan and the dependence between breasts. Restricted cubic splines with 4 knots to the number of FAST MRI scans read overall and per reading session (batch size) were also included in the models to assess whether the readers performance improved during the assessment task. The agreement between readers and the true outcome was assessed using Cohen’s κ coefficient, to account for the probability of agreement occurring by chance. The diagnostic odds ratio was determined as a measure of overall diagnostic accuracy independent of prevalence( 22 ). Classifications 4 and 5 were considered indicative of cancer, and classifications 1–3 considered a normal result. Interpretation times were compared across reader groups (Wilcoxon rank-sum). Results There were 43 participants from 7 sites, 22 with previous experience of reading fpMRI (Group 1) and 21 new to reading MRI (Group 2). Eighteen participants (11 from Group 1 and 7 from Group 2) had previously undertaken in-person FAST MRI training as part of a research study (“Attended”) and the remaining participants in each group had not, (“Not attended”)(11,13). None of the data presented in the results section overlaps with that of the previous two studies. All participants completed the training, including reading the formative assessment task (test set) of 125 FAST MRI scans (250 breasts). Individual readers’ opinions for 4 scans failed to register due to a technical error, giving a total of 10,746 reads. Figure 1 shows the flow chart of reader recruitment ( Figure 1 ) and Table 1 details participants’ professional experience ( Table 1 ). Table 1: Demographics of participant mammogram readers Group 1* Group 2* Number of participants 22 21 Professional Title** Advanced Practitioner 0 7 Consultant Radiographer 0 7 Breast Clinician 0 4 Radiologist Associate Specialist 0 1 Consultant Radiologist 22 2 Professional Experience Number of years interpreting mammograms: median (range) 9 (1-22) 7 (1-28) Number of mammograms interpreted each year: median (range) 6000 (3000-12000) 7000 (4000-10000) Participant readers who interpret digital breast tomosynthesis (DBT) in normal clinical practice 22 15 Number of years interpreting breast MRI: median (range) 8 (1-25) N/A Number of full protocol breast MRI scans interpreted each year: median (range) 135 (30-450) N/A Total numbers of participant readers who previously attended in person FAST MRI training 11 7 * Reader group: group 1 = mammogram readers with experience of fpMRI interpretation in their usual clinical practice, group 2 = mammogram readers with no previous experience of breast MRI interpretation in their clinical practice. **Professional titles in UK: Screening mammograms within the NHS Breast Screening Programme are interpreted by multidisciplinary healthcare professionals trained in mammogram interpretation. Their performance is subject to continuous audit through the UK Breast Screening Information System that produces individual real-life performance data over rolling 3-year periods (35). “Consultant Radiologist” and “Breast Clinician” are titles held by medical doctors. Consultant Radiologists are registered on the General Medical Council’s Specialist Register following Completion of Specialist Training (5 years) with standards and curriculum set by the Royal College of Radiologists (RCR). The Association of Breast Clinicians launched the Credential in Breast Disease Management for Breast Clinicians, jointly with the RCR, in 2019, to standardise and formalise training for Breast Clinicians across the UK (3-year training programme)(36). “Advanced Practitioners” and “Consultant Radiographers” are experienced, registered healthcare practitioners, typically mammographers, who have additionally completed specialist training, underpinned by a master’s level award or equivalent to support their professional practice within the NHS (37). Figure 1: Flow diagram detailing participation in FAST MRI interpretation e-learning study Per-breast analysis The per-breast analysis comparing readers’ MRI classification with the true outcome (cancer or normal) showed an overall sensitivity of 83% (95%CI 81-84%; 1994/2408) and specificity of 94% (95%CI 93–94%; 7806/8338). Readers with experience of fpMRI interpretation (Group 1) showed similar sensitivity (1034/1232; 84%; 95%CI 82-86%) but slightly higher specificity (4031/4266; 94%; 95%CI 94–95%) than readers without fpMRI experience (Group 2) (sensitivity = 82%; 95%CI 79–84% (960/1176) p=0.14; specificity = 93%; 95%CI 92–93% (3775/4072) p = 0.001) ( Table 2 ). Those readers that had previously completed in-person FAST MRI interpretation training (“Attended”) had a significantly higher overall sensitivity (88%; 95% CI 85-91%) than those that had not attended (80%; 95% CI 78-82%, p<0.0001), but significantly lower specificity (92%; 95% CI 92-93% compared to 94%; 95% CI 94-95%, p=0.003), irrespective of group ( Table 2 ). The diagnostic accuracy results are summarised in Figure 2, which plots readers’ accuracy in the receiver operating characteristic space by group and by whether previously attended in person FAST MRI training ( Figure 2 ). Figure 2: Diagnostic accuracy in the receiver operating characteristic (ROC) space Point estimates of accuracy for individual readers in ROC space Plot of accuracy in ROC space for each group and attendance or non-attendance with error bars for 95%CIs Both the inter-reader agreement (kappa) of readers with the true outcome and the diagnostic odds ratio (DOR) were higher for Group 1 (kappa 0.77 (95%CI: 0.76-0.80), DOR 89.58 (95% CI 73.26-109.52) than Group 2 (0.73 (0.70-0.75) and 56.49 (46.76-68.25)) and tended to be higher for those participants that had attended previous FAST MRI training (kappa 0.77 (95%CI: 0.75-0.79), DOR 87.95 (95% CI 70.27-110.08) compared to those participants that had not previously completed FAST MRI training (kappa 0.74 (95%CI: 0.72-0.76), DOR 62.98 (95% CI 52.77-75.16) ( Table 2 ). Table 2: Readers’ diagnostic accuracy by group* and by attendance or non-attendance at previous in-person FAST-MRI interpretation-training Category Measure Concordance (Accuracy) True positive rate (Sensitivity) True negative rate (Specificity) Kappa (95% CI) Diagnostic odds ratio (95% CI) All readers 9800/10746 (91%) 1994/2408 (83%) 7806/8338 (94%) 0.75 (0.74-0.77) 70.67 (61.59-81.09) Reader group* Group 1* 5065/5498 (92%) 1034/1232 (84%) 4031/4266 (94%) 0.77 (0.76-0.80) 89.58 (73.26-109.52) Group 2* 4735/5248 (90%) 960/1176 (82%) 3775/4072 (93%) 0.73 (0.70-0.75) 56.49 (46.76-68.25) Attendance at previous in person FAST MRI interpretation training Attended 4123/4500 (92%) 876/1008 (87%) 3247/3492 (93%) 0.77 (0.75-0.79) 87.95 (70.27-110.08) Not attended 5677/6246 (91%) 1118/1400 (80%) 4559/4846 (94%) 0.74 (0.72-0.76) 62.98 (52.77-75.16) Reader group* and attendance at previous in person FAST MRI interpretation training Group 1* Attended 1613/1750 (92%) 533/616 (87%) 2002/2134 (94%) 0.78 (0.75-0.81) 97.40 (72.83-130.25) Group 1* Not a ttended 2530/2748 (92%) 501/616 (81%) 2029/2132 (95%) 0.77 (0.74-0.80) 85.82 (64.65-113.93) Group 2* Attended 1588/1750 (91%) 343/392 (88%) 1245/1358 (92%) 0.75 (0.71-0.78) 77.12 (54.03-110.09) Group 2* Not a ttended 3147/3498 (90%) 617/784 (79%) 2530/2714 (93%) 0.71 (0.69-0.74) (40.48-63.76) * Reader group: group 1 = experience of fpMRI interpretation in their usual clinical practice, group 2 = no previous experience of breast MRI interpretation in their clinical practice Plotting the Learning Curve Readers’ sensitivity remained fairly stable during the test set reading process (formative assessment task) (p=0.24) and this effect was similar for both groups (interaction p=0.30) and whether or not they had previously completed FAST MRI training (interaction p=0.97). However, specificity was significantly affected by the number of scans read in the formative assessment task (p<0.001) and this effect differed, depending on whether readers had attended previous FAST MRI training or not (interaction p=0.01) but not between groups (interaction p=0.08). The predicted specificity curves for readers that had attended previous FAST MRI training reached a peak after 75 reads but continued to increase for those that had not attended previous FAST MRI training, with group 1 readers having significantly higher specificity than group 2 (p=0.003) ( Figure 3 ). Figure 3: Changes in reader specificity with number of test-set FAST MRI scans read over time Multi-level generalised mixed model using restricted cubic splines with 4 knots to the number of scans read over time by attendance or non-attendance at previous FAST-MRI training and by group Accuracy (concordance with the true outcome) changed significantly by the number of FAST MRI scans read in the test set reading process (formative assessment task) (p=0.002) and differed depending on whether or not a reader had previously attended FAST MRI training (interaction p=0.02) but was similar for both groups (interaction p=0.36). Accuracy was significantly higher for Group 1 than Group 2 overall (p=0.001) and reached a peak after 75 reads for those readers that had previously attended FAST MRI training, as seen with the results for specificity ( Figure 4 ). Figure 4: Changes in concordance with the true outcome by number of test-set FAST-MRIs read over time Multi-level generalised mixed model using restricted cubic splines with 4 knots to the number of scans read over time by attendance or non-attendance at previous FAST-MRI training and by group Reading pattern: batch size Readers read the formative assessment task of 125 FAST MRI scans in a median of 2 batches (range 1- 8 batches) with a median of 32 scans read per batch (range 1-125 scans/batch). The readers that did not attend the previous FAST MRI in person training tended to complete this current training in fewer batches and hence had larger batch sizes than those that had attended previous training ( Table 3 ). Table 3: Batch reading pattern by group* and by attendance or non-attendance at previous in-person FAST-MRI interpretation-training Group 1* Group 1* Group 2* Group 2* Attended Not attended Attended Not attended Number of batches Median (IQR) 2 (2-4) 2 (1-4) 3 (3-5) 1 (1-3) Range 1-8 1-5 2-8 1-7 Number of FAST MRI scans read in a batch Median (IQR) 31 (15-58) 33 (23-63) 30 (14-41) 50 (19-125) Range 9-125 7-125 1-84 1-125 * Reader group: group 1 = experience of fpMRI interpretation in their usual clinical practice, group 2 = no previous experience of breast MRI interpretation in their clinical practice Accuracy (concordance with the true outcome) significantly changed depending on the number of reads within a batch (p=0.02) but in a similar manner for both groups (interaction p=0.53) and for whether or not a reader had previously attended FAST MRI training (interaction p=0.78). Accuracy tended to worsen after 50 FAST MRI scans were read within a batch for both groups ( Figure 5 ). Figure 5: Changes in concordance with the true outcome by number of test-set FAST-MRIs read per batch Multi-level generalised mixed model using restricted cubic splines with 4 knots fitted to the rank order of FAST-MRI scans read per batch by reader group Similarly, specificity significantly changed with the number of reads within a batch (p= 0.0001) for both groups (interaction p=0.18). Sensitivity, although not significantly dependent, tended to worsen with increasing number of MRI scans read within a batch (p= 0.08) and this was similar for both groups (interaction p=0.91). Time taken to interpret The median time taken for the individual readers to interpret each FAST MRI scan was 22 seconds less for Group 1 (median 56 seconds, range 8-47466 seconds) than for Group 2 (median 78 seconds, 14-22830 seconds, p <0.0001). Seven records had a total time of more than 1000 seconds. Discussion In comparison with previous FAST MRI training results from the two studies that used in-person versions of the same standardised training programme and assessment dataset (delivered as one-to-one( 13 ) or small group( 11 ) training), FAST MRI readers in the current study (trained with remote e-learning and automated formative assessment) achieved higher specificity (94%; 95%CI 93–94 vs. 87%; 85–89 (one-to-one training)( 13 ) and 86%; 85–86 (small group training)( 11 )), and higher overall diagnostic accuracy (reader agreement with the true outcome)(kappa 0.75; 0.74–0.77 vs. 0.69; 0.65–0.72( 13 ) and 0.63; 0.61–0.65( 11 ), and DOR 70.67 (61.59–81.09) vs. 48.30 (35.12–66.44)( 13 ) and 35.49 (30.87–40.81)( 11 )). There was, however, a trend for lower sensitivity (83%; 81–84 vs. 88%; 84–91( 13 ) and 86% 84–87( 11 )) at cancer detection. Specificity improved during the test set interpretation assessment task, similarly for both groups but the effect differed depending on whether readers had attended previous training sessions. Whilst readers that had attended previous FAST MRI training tended to reach a peak after 75 reads, the specificity continued to improve for those that had not attended previous FAST MRI training. Similar learning curves were also seen for concordance with the true outcome. Sensitivity remained stable over all reads. Specificity is arguably the most important diagnostic accuracy parameter to optimise in the context of a breast screening tool that is designed for women with a low pre-test probability. This is because small changes in specificity can have a large effect on the number of false positive recalls in a population screening programme, with each recall causing harm to the woman screened and also incurring a financial and workforce cost( 14 – 16 ). The specificity for mammography in the NHS Breast Screening Programme (NHSBSP), interpreted by double-reading, is 96%( 17 ). The specificity achieved at single read of an enriched test set of FAST MRI scans by readers in the current study could be considered comparable (94%). These results demonstrate that the inclusion of immediate feedback for each scan during test set interpretation in FAST MRI reader training optimised specificity whilst maintaining high levels of sensitivity, which would suit a screened population with low pre-test probability. Achievement of reporting benchmarks for fpMRI Two days of standardised FAST MRI interpretation-training, undertaken as remote e-learning, enabled NHSBSP mammogram-readers, both those experienced in fpMRI interpretation (Group 1) and novice MRI readers (Group 2), to achieve, at single read of an enriched dataset, benchmarks set for fpMRI interpretation in practice by the American College of Radiology’s Breast Imaging Reporting and Data System (BI-RADS) for both sensitivity (Groups 1 (84%) and 2 (82%) vs. >80% BI-RADS benchmark( 23 )) and specificity (Groups 1 (94%) and 2 (93%) vs. >85% BI-RADS benchmark( 23 )). Of 43 participants, the two-day remote e-learning programme was sufficient for 43/43 (100%) to achieve specificity above the 85% BI-RADS benchmark and for 33/43 (77%) to achieve sensitivity above the 80% BI-RADS benchmark. Novice MRI readers (Group 2) achieved similar sensitivity to experienced fpMRI readers (Group 1) (p = 0.14) but lower specificity (p = 0.001) although specificity differed between groups by only one percentage point (Group 1: 94% and Group 2: 93%). Comparison between the performance of those who had previously attended in-person FAST MRI interpretation training and those who had not Whilst the reader agreement with the true outcome (kappa) and the DOR did not differ significantly between the readers who had previously attended in-person FAST MRI interpretation-training (11/22 in Group 1 and 7/21 in Group 2) and those who had not, the sensitivity for cancer detection was higher and the specificity lower for the “attended” cohort than for the “not attended” cohort. Looking at the individual performance, during a previous study( 11 ), of the 14 participants of the current study who had attended previous small group training, 8 of these participants had a sensitivity in the top 9 sensitivities of participants in the previous study and none were in the bottom 7 sensitivities( 11 ). Additionally, 8 of these participants had specificity in the bottom 12 for specificity in the previous study and 3 were in the top 11 specificities( 11 ). Therefore, self-selection bias could have contributed to the within group significant differences of sensitivity and specificity found for attendance vs. non-attendance at previous in person training. Literature comparison – the effect of batch size on diagnostic performance The Co-Ops Study assessed the effect of reading practice, including batch size, on reader diagnostic performance in mammography within the NHSBSP and demonstrated increased specificity with increased batch size up to 40 mammograms per batch with the trend continuing in longer batches( 24 ). The current study, whilst it showed a trend for increasing specificity with batch size up to 50 FAST MRI scans per batch and decreasing sensitivity with increasing batch size, also demonstrated that concordance with the true outcome (as a measure of overall accuracy) tended to worsen when more than 50 scans were read within one batch. This accords with results from a study of 2,937,312 mammogram reads that demonstrated both small increases in specificity and small decreases in sensitivity for mammograms read at later positions within a batch. The authors of the study suggested that optimal batch-size for reading mammograms could be 60–70 reads per batch( 25 ). One possible explanation for the optimal batch size for FAST MRI (50 scans per batch) being smaller than that suggested for mammograms could be the difference in complexity between reading FAST MRI scans and mammograms. Reading FAST MRI scans in the current study could more quickly cause fatigue for readers than reading mammograms because FAST MRI reading format requires more images to be reviewed per scan than for a mammogram. However, the reading format of digital breast tomosynthesis (DBT)(2D plus stack of reconstructed slabs) has a similar complexity to that of FAST MRI (MIP plus stack of slices) and although we could find no study that reported the effect of reading batch size on the diagnostic accuracy of DBT, evidence of increasing reader fatigue during the process of reading a batch of 40 DBT scans has been reported( 26 ). Literature comparison – reading times The reading times achieved by readers in this study (56 and 78 seconds for Groups 1&2) were longer than times reported for NHSBSP mammogram readers to interpret mammograms (35 and 76 seconds ( 27 , 28 )) and about half that reported for NHSBSP mammogram readers to interpret DBT (2.81 minutes)( 26 ). However, evidence is emerging that various AI strategies may reduce reading times for DBT without affecting accuracy( 29 , 30 ). In the future similar approaches may prove valuable for FAST MRI. Limitations of the current study Readers who had previously attended FAST MRI interpretation training had interpreted the same test set of 125 FAST MRI scans during the previous study. However, since they had not previously seen the ground truth (true outcome) of the scans in the test set at any time, and there was an average time interval of 24 months (range 17–30 months) between reading the test set in the two studies, it is unlikely that their diagnostic performance was affected by this. The test set was read outside normal clinical practice and therefore reader performance is likely to have been subject to a laboratory effect( 31 ). Readers were free to self-select batch length when reading the test set assessment task. Therefore, our conclusions on optimal batch size could potentially have been confounded through self-selection bias. However, similar results were seen with the subset of readers who completed all 125 scans of the test set in a single batch (7 from Group 1 and 8 from Group 2) ( Appendix 3 ), suggesting the effect of self-selection bias, although unquantifiable, is likely to be small. Implications of the research The single reading performance at FAST MRI achieved by experienced (Group 1) and novice (Group 2) readers in the current study, reading an enriched dataset, compares well with published figures for diagnostic performance at fpMRI for radiologists experienced in breast MRI interpretation in community screening practice in the USA (Breast Cancer Surveillance Consortium (BCSC)( 32 ): sensitivity: 84% (Group 1) and 82% (Group 2) vs. 81% (BCSC), and specificity: 94% (Group 1) and 93% (Group 2) vs. 83% (BCSC). Double reading is the current standard for reading within NHSBSP and has the potential to further improve diagnostic accuracy for mammogram readers who have undertaken standardised FAST MRI interpretation training. FAST MRI was designed as a screening test that would provide the high sensitivity of fpMRI for aggressive breast cancers at a fraction of the cost through shorter acquisition and reading times( 4 ), with the intention that it could be used to screen a wider population than currently benefit from screening with fpMRI( 33 , 34 ). However, for mass screening tests, specificity is arguably the most important metric( 14 – 16 ) and trials of breast MRI (scans single read by expert fpMRI readers) for women with dense breasts, but otherwise at population risk of breast cancer, have reported results with lower specificity (86.7%( 3 ) and 92.6%( 1 )) than sensitivity (95.7%( 3 ) and 95.2%( 1 )). The specificity achieved for FAST MRI by mammogram readers in the current study following 2 days of standardised training (94%) compares well with the results from both these MRI screening trials and approaches the specificity of mammography achieved with double reading within the NHSBSP for population screening (96%)( 17 ). Conclusions Future trials of FAST MRI will benefit from standardising the training, assessment, and credentialing of FAST MRI readers. Providing formative assessment as part of interpretation-training and optimising reading batch size can increase specificity and provide high diagnostic accuracy at FAST MRI. The diagnostic accuracy achieved at single read by NHSBSP mammogram readers in this study suggests that two-day standardised FAST MRI remote e-learning, that includes formative assessment using an enriched dataset, could form the basis for FAST MRI interpretation-training for mammogram readers who wish to participate as readers in future FAST MRI trials and clinical practice. Credentialling readers could be accomplished using BI-RADS benchmarks of performance( 23 ) as cut offs for sensitivity and specificity achieved by readers in the assessment task. Abbreviations ACR American College of Radiology ANOVA analysis of variance AUC Area under a curve BCSC Breast Cancer Screening Consortium BSIS Breast Screening Information Service BI-RADS American College of Radiology’s Breast Imaging Reporting and Data System CI Confidence interval DCIS Ductal carcinoma in-situ DOR Diagnostic Odds Ratio FAST MRI First post contrast subtracted images (abbreviated breast MRI) fpMRI Full protocol breast Magnetic Resonance Imaging IRAS Integrated Research Application System for applications to the Health Research Authority and the Research Ethics Committee ISRCTN International Standard Randomised Controlled Trial Number (however, over the years the scope of the registry has widened beyond randomised controlled trials to include any study designed to assess the efficacy of health interventions in a human population.) MIP Maximum intensity projection image MRI Magnetic Resonance Imaging NHSBSP National Health Service (United Kingdom) Breast Screening Programme PERFORMS Personal Performance in Mammographic Screening REC Research Ethics Committee ROI region of interest UK United Kingdom of Great Britain and Northern Ireland USA United States of America Declarations Ethics approval and consent to participate : In accordance with the Declaration of Helsinki on research involving human participants, this study was reviewed and approved by the Health and Care Research Wales Ethics Committee REC:21/HRA/4543 and by the Health Research Authority (IRAS:301714).[LC1] The study was prospectively registered (ISRCTN:16624917). All participants gave informed consent (written) to their participation in the study. Consent for publication : Consent to participate in this study included consent for publication of non-identifiable data and was given by all participants. Availability of data and materials: The dataset generated and analysed during the current study is not yet publicly available because it is currently being developed into a publicly shareable format. Instead, it is available from the corresponding author on reasonable request. Competing interests: Other than the funding sources declared below, the authors declare that they have no competing interests. Funding: This manuscript presents independent research funded by Health Education England through a Bursary for Educational Innovation from the National Breast Imaging Academy (Mapping the learning curve of NHS Breast Screening Programme (NHSBSP) mammogram readers new to the interpretation of a shortened magnetic resonance imaging scan (MRI) of the breast called FAST MRI to support the delivery of a future multicentre trial of FAST MRI versus mammogram for breast cancer screening). The electronic standardised teaching and assessment tools that were used within the new FAST MRI e-learning programme described in this manuscript were originally developed during work funded by the National Institute for Health Research (Research for Patient Benefit (RfPB), Refinement and piloting of a training programme within the NHS Breast Screening Programme (NHSBSP) workforce of image readers to enable standardised interpretation of a shortened magnetic resonance imaging scan (MRI) of the breast called FAST MRI to support the delivery of a future multicentre trial of FAST MRI versus mammogram for breast cancer screening, PB-PG-1217-20008)(11). The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research (NIHR) or the Department of Health and Social Care. Author STP is supported by an NIHR Career Development Fellowship (CDF – 2016-09-018). The views expressed in this manuscript are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care. Authors’ contributions: LJ, AM and JD contributed substantially to the conception of the work. LJ, JD, EOF, RG, AM, STP, PE and SV had substantial input to the study design. PE, MHB, LJ and RG contributed substantially to the creation of new software used in the study. TT, PE, AM, LJ, RG and SMK had substantial input to the acquisition of data, while AM and PE conducted the data analysis and LJ, AM, STP, SV, EOF and JD contributed substantially to the data interpretation. LJ, AM and TT drafted the work and subsequently, with additional help from STP, SV, RG, and EOF, substantially revised it. The corresponding author is LJ. All authors have approved the submitted version of this manuscript and have agreed both to be personally accountable for the author’s own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, including ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. Acknowledgements This study was performed on behalf of the FAST MRI Study Group [LC2] which at the time of this study, in addition to the authors comprised: Clare Alison, Karen Atkinson, Miklos Barta, Gemini Beckett, Claudia Betancourt, Julie Bramwell, Holly Brown, Helen Burt, Louise Cann, Nick Carter, Claire Cartledge, Jane Ceney, Gillian Clark, Eleanor Cornford, Elizabeth Cullimore, Siân Curtis, Diana Dalgliesh, Jonathon Delve, Sarah Doyle, Alison Duncan, Holly Elbert, Sarah Fearn, Christopher Foy, Zsolt Friedrich, Hesam Ghiasvand, John Gifford, Dagmar Godden, Zoe Goldthorpe, Sandra Gomes, Narayan Aradhana Goud, Rosie Gray, Sam A. Harding, Kristin Henning, Lucinda Hobson, Claire Hulme, Paula Hynam, El Sanharawi Imane, Emma Jackson, Asif Jaffa, Ragini Jhalla, Margaret Jenkin, Thomas William Jones, Nahid Kamangari, Vandana Kaur, Beckie Kingsnorth, Katherine Klimczak, Elisabeth Kutt, Karen Litton, Simon Lloyd, Iain Lyburn, Anjum Mahatma, Anna Mankelow, Helen Massey, Helen Matthews, Karis McFeely, Clare McLachlan, Sarah McWilliams, Shahrooz Mohammadi, Alice Moody, Elizabeth Muscat, Sreenivas Muthyala, Sarah Perrin, Alison Peters, Alice Pocklington, Elizabeth Preston, Jasvinder Rai, Jo Robson, Corri Salter, Toni Scanlon, Anuma Shrestha, Richard Sidebottom, Mary Sinclair, Sravya Singamaneni, Jim Steel, Lesley Stephenson, Sam Stewart-Maggs, Cheryl Stubbs, Michelle Taylor, Victoria Taylor, Olivia Taylor-Fry, Erika Toth, Matthew Trumble, Alexandra Valencia, Frances Vincent, Anna Wang, Lucy Warren, Sharon Watkin, Sue Widdison, Jennifer Williams and Jennifer Wookey. The authors wish to thank the Breast Unit Support Trust (BUST) and the Independent Cancer Patients’ Voice (ICPV) charities and the NIHR Research Design Service (RDS) for their invaluable support. References Bakker MF, De Lange S V., Pijnappel RM, Mann RM, Peeters PHM, Monninkhof EM, et al. Supplemental MRI screening for women with extremely dense breast tissue. New England Journal of Medicine [Internet]. 2019;381(22):2091–102. Available from: https://www.nejm.org/doi/full/ 10.1056/NEJMoa1903986 Veenhuizen SG, De Lange S V., Bakker MF, Pijnappel RM, Mann RM, Monninkhof EM, et al. Supplemental Breast MRI for Women with Extremely Dense Breasts: Results of the Second Screening Round of the DENSE Trial. Radiology [Internet]. 2021;00:1–9. Available from: https://doi.org/10.1148/radiol.2021203633 Comstock CE, Gatsonis C, Newstead GM, Snyder BS, Gareen IF, Bergin JT, et al. Comparison of Abbreviated Breast MRI vs Digital Breast Tomosynthesis for Breast Cancer Detection Among Women With Dense Breasts Undergoing Screening. JAMA - Journal of the American Medical Association [Internet]. 2020;323(8):746–56. Available from: https://jamanetwork.com/journals/jama/fullarticle/2761645 Kuhl CK, Schrading S, Strobel K, Schild HH, Hilgers RD, Bieling HB. Abbreviated breast Magnetic Resonance Imaging (MRI): First postcontrast subtracted images and maximum-intensity projection - A novel approach to breast cancer screening with MRI. Journal of Clinical Oncology [Internet]. 2014;32:2304–10. Available from: https://ascopubs.org/doi/10.1200/JCO.2013.52.5386 Vinnicombe S, Harvey H, Healy N. Introduction of an abbreviated breast MRI service in the UK as part of the BRAID trial: practicalities, challenges and future directions. Clin Radiol [Internet]. 2021;76(6):427–33. Available from: https://doi.org/10.1016/j.crad.2021.01.020 Geach R, Jones LI, Harding SA, Marshall A, Taylor-Phillips S, Mckeown-keegan S, et al. The potential utility of abbreviated breast MRI (FAST MRI) as a tool for breast cancer screening: a systematic review and meta-analysis. Clin Radiol [Internet]. 2021;76:154.e11-154.e22. Available from: https://doi.org/10.1016/j.crad.2020.08.032 Baxter GC, Selamoglu A, Mackay JW, Bond S, Gray E, Gilbert FJ. A meta-analysis comparing the diagnostic performance of abbreviated MRI and a full diagnostic protocol in breast cancer. Clin Radiol [Internet]. 2021;76:154.e23-154.e32. Available from: https://doi.org/10.1016/j.crad.2020.08.036 Grimm LJ, Mango VL, Harvey JA, Plecha DM, Conant EF. Implementation of Abbreviated Breast MRI for Screening: AJR Expert Panel Narrative Review. American Journal of Roentgenology [Internet]. 2022;218(2):202–12. Available from: https://doi.org/10.2214/AJR.21.26349 Marshall HN, Plecha DM. Setting up an Abbreviated Breast MRI Program: Our Two-year Implementation Experience. J Breast Imaging [Internet]. 2020;2(6):603–8. Available from: https://doi.org/10.1093/jbi/wbaa060 Lee CS, Moy L, Hughes D, Golden D, Bhargavan-Chatfield M, Hemingway J, et al. Radiologist characteristics associated with interpretive performance of screening mammography: A national mammography database (nmd) study. Radiology [Internet]. 2021 Sep 1 [cited 2023 Jun 12];300(3):518–28. Available from: https://doi.org/10.1148/radiol.2021204379 Jones LI, Marshall A, Elangovan P, Geach R, McKeown-Keegan S, Vinnicombe S, et al. Evaluating the effectiveness of abbreviated breast MRI (abMRI) interpretation training for mammogram readers: a multi-centre study assessing diagnostic performance, using an enriched dataset. Breast Cancer Research [Internet]. 2022;24(1). Available from: https://doi.org/10.1186/s13058-022-01549-5 Harding S, Geach R, Jones L. The use of ‘Think-Out-Loud’ methodology in the development of teaching materials for abbreviated breast Magnetic Resonance Imaging scan (FAST MRI) interpretation, and a comparison of the learning experience of two reader cohorts. Eur J Radiol Open [Internet]. 2019;6(May):220–4. Available from: https://doi.org/10.1016/j.ejro.2019.06.002 Jones LI, Geach R, Harding SA, Foy C, Taylor V, Marshall A, et al. Can mammogram readers swiftly and effectively learn to interpret first post-contrast acquisition subtracted (FAST) MRI, a type of abbreviated breast MRI?: a single centre data-interpretation study. Br J Radiol [Internet]. 2019;92(July):20190663. Available from: https://doi.org/10.1259/bjr.20190663 Taylor-Phillips S, Seedat F, Kijauskaite G, Marshall J, Halligan S, Hyde C, et al. UK National Screening Committee’s approach to reviewing evidence on artificial intelligence in breast cancer screening. Lancet Digit Health [Internet]. 2022;4(7):e558–65. Available from: https://doi.org/10.1016/S2589-7500(22)00088-7 Cole P MA. Basic issues in population screening for cancer. J Natl Cancer Inst. 1980;64(5):1263–7. Ferrante Di Ruffano L, Hyde CJ, McCaffery KJ, Bossuyt PMM, Deeks JJ. Assessing the value of diagnostic tests: A framework for designing and evaluating trials. BMJ (Online) [Internet]. 2012;344(7847). Available from: https://doi.org/10.1136/bmj.e686 Burnside ES, Vulkan MSD, Blanks RG, Duffy SW. Association between Screening Mammography Recall Rate and Interval Cancers in the UK Breast Cancer Service Screening Program: A Cohort Study. Radiology [Internet]. 2018;288:47–54. Available from: https://doi.org/10.1148/radiol.2018171539 Black P, Wiliam D. “In Praise of Educational Research”: Formative assessment. Br Educ Res J [Internet]. 2003;29(5):623–37. Available from: https://doi.org/10.1080/0141192032000133721 Public Health England. Technical guidelines for magnetic resonance imaging (MRI) for the surveillance of women at higher risk of developing breast cancer (NHSBSP Publication No 68) [Internet]. Gov.Uk. 2012. Available from: https://www.gov.uk/government/publications/nhs-breast-screening-using-mri-with-higher-risk-women Taylor K, Britton P, O’Keeffe S, Wallis MG. Quantification of the UK 5-point breast imaging classification and mapping to BI-RADS to facilitate comparison with international literature. British Journal of Radiology [Internet]. 2011;84(1007):1005–10. Available from: https://doi.org/10.1259/bjr/48490964 Donner A, Rotondi MA. Sample size requirements for interval estimation of the kappa statistic for interobserver agreement studies with a binary outcome and multiple raters. International Journal of Biostatistics [Internet]. 2010 [cited 2023 Jun 18];6(1). Available from: https://doi.org/10.2202/1557-4679.1275 Suzuki S. Conditional Relative Odds Ratio and Comparison of Accuracy of Diagnostic Tests Based on 2×2 Tables. J Epidemiol [Internet]. 2006 [cited 2023 Oct 1];16(4):145–53. Available from: https://doi.org/10.2188/jea.16.145 Sickles E, D’Orsi C. ACR BI-RADS® Follow-up and Outcome Monitoring. In: ACR BI-RADS® Atlas, Breast Imaging Reporting and Data System [Internet]. 5th ed. Reston, VA: American College of Radiology; 2013. Available from: https://www.acr.org/Clinical-Resources/Reporting-and-Data-Systems/Bi-Rads/Permissions Taylor-Phillips S, Wallis MG, Jenkinson D, Adekanmbi V, Parsons H, Dunn J, et al. Effect of using the same vs different order for second readings of screening mammograms on rates of breast cancer detection: A randomized clinical trial. JAMA - Journal of the American Medical Association [Internet]. 2016;315(18):1956–65. Available from: https://doi.org/10.1001/jama.2016.5257 Backmann HA, Larsen M, Danielsen AS, Hofvind S. Does it matter for the radiologists’ performance whether they read short or long batches in organized mammographic screening? Eur Radiol [Internet]. 2021;31:9548–55. Available from: https://doi.org/10.1007/s00330-021-08010-9 Chen Y, Sudin ES, Partridge GJ, Taib AG, Darker IT, Phillips P, et al. Measuring reader fatigue in the interpretation of screening digital breast tomosynthesis (DBT). British Journal of Radiology [Internet]. 2023 Feb 1 [cited 2023 May 17];96(1143). Available from: https://doi.org/10.1259/bjr.20220629 Partridge G, Darker I, James J, Michell M, Chen Y. How long does it take to read a mammogram. In: Symposium mammographicum abstract booklet [Internet]. 2023 [cited 2023 Oct 2]. p. 13–13. Available from: https://sympmamm.org.uk/wp-content/uploads/2023/07/Symposium-Mammographicum-Abstracts-2023.pdf Taylor-Phillips S, Wallis M, Gale AG. Should previous mammograms be digitised in the transition to digital mammography ? Breast [Internet]. 2009 [cited 2023 Oct 14];19:1890–6. Available from: https://doi.org/10.1007/s00330-009-1366-x Dahlblom V, Dustler M, Tingberg A, Zackrisson S. Breast cancer screening with digital breast tomosynthesis: comparison of different reading strategies implementing artificial intelligence. Eur Radiol [Internet]. 2023;33:3754–65. Available from: https://doi.org/10.1007/s00330-022-09316-y Raya-Povedano JL, Romero-Martín S, Elías-Cabot E, Gubern-Mérida A, Rodríguez-Ruiz A, Álvarez-Benito M. AI-based Strategies to Reduce Workload in Breast Cancer Screening with Mammography and Tomosynthesis: A Retrospective Evaluation. Radiology [Internet]. 2021 Jul 1 [cited 2023 Jun 13];300(1):57–65. Available from: https://doi.org/10.1148/radiol.2021203555 Gur D, Cohen CS, Hakim CM, Hardesty LA, Ganott MA, Perrin RL, et al. The “Laboratory Effect”: Comparing Radiologists’ Performance and Variability during Prospective Clinical and Laboratory Mammography. Radiology [Internet]. 2008;249(1):47–53. Available from: https://doi.org/10.1148/radiol.2491072025 Lee JM, Ichikawa L, Valencia E, Miglioretti DL, Wernli K, Kerlikowske K, et al. Performance Benchmarks for Screening Breast MR Imaging in Community Practice. Radiology [Internet]. 2017;285(1):44–52. Available from: https://doi.org/10.1148/radiol.2017162033 Saadatmand S, Obdeijn IM, Rutgers EJ, Oosterwijk JC, Tollenaar RA, Woldringh GH, et al. Survival benefit in women with BRCA1 mutation or familial risk in the MRI screening study (MRISC). Int J Cancer [Internet]. 2015;137(7):1729–38. Available from: https://doi.org/10.1002/ijc.29534 Evans DG, Harkness EF, Howell A, Wilson M, Hurley E, Holmen MM, et al. Intensive breast screening in BRCA2 mutation carriers is associated with reduced breast cancer specific and all cause mortality. Hered Cancer Clin Pract [Internet]. 2016;14(8):1–8. Available from: http://dx.doi.org/10.1186/s13053-016-0048-3 Chen Y, James JJ, Michalopoulou E, Darker IT, Jenkins J. Performance of Radiologists and Radiographers in Double Reading Mammograms: The UK National Health Service Breast Screening Program. Radiology [Internet]. 2023 Jan 1 [cited 2023 Jun 12];306(1):102–9. Available from: https://doi.org/10.1148/radiol.212951 National Breast Imaging Academy. https://nationalbreastimagingacademy.org/breast-clinicians/about-breast-clinicians/breast-clinician-credential/ . 2019. Breast Clinician Credential. NHS Health Education England. https://advanced-practice.hee.nhs.uk . 2023. Advanced Practitioner Training. Additional Declarations No competing interests reported. Supplementary Files Additionalfile1.pdf Additional File 1 (.pdf): Specification of the FAST MRI protocol and composition of the assessment test-set The specification of the FAST MRI protocol used in the current study and the composition of the assessment test set used in current study have been previously published. They are reproduced here in line with the copyright policy of the journal in which they were previously published. Additionalfile2.pdf Additional File 2 (.pdf): Standardised FAST MRI interpretation-training programme The FAST MRI interpretation-training programme delivered as e-learning in the current study was adapted from a previously developed, standardised, in-person interpretation-training programme described in a previous publication. Details of the training programme have been reproduced here in line with the copyright policy of the journal in which they were previously published. Additionalfile3.pdf Additional File 3 (.pdf): Comparison graphic, included to inform discussion of our conclusions on optimal batch size The relationship of accuracy with batch size of the subset of readers who completed reading the assessment test set in a single batch of 125 FAST MRI scans is presented (for comparison with Figure 5 - the equivalent graphic for all readers). The information is presented as a graphic entitled: Changes in concordance with the true outcome (accuracy) by scan position within a batch, and by reader group*, showing only the readers that completed reading the assessment test set of 125 FAST MRI scans in a single batch (multi-level generalised mixed model using restricted cubic splines with 4 knots fitted to the rank order of FAST MRI scans read per batch). Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 25 Feb, 2024 Reviews received at journal 19 Feb, 2024 Reviewers agreed at journal 05 Feb, 2024 Reviewers invited by journal 05 Feb, 2024 Editor assigned by journal 22 Jan, 2024 Submission checks completed at journal 21 Jan, 2024 First submitted to journal 20 Jan, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-3881738","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":268474579,"identity":"037f90b9-1654-4c85-9b77-d8740dcd7cf0","order_by":0,"name":"Lyn I Jones","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA2ElEQVRIiWNgGAWjYBACAxBRASLYG9hAVAIQMxPWIgEieA6wMRwgTYtEArFaJHIPMEhU3JGTn/n22OMPDHfy5Bt4jA3wa8lLYJA488zY4HZeusEBhmfFBgd4jBPwa8kxYJBsO5y4QTrHTOIAA5DBwGN8gBgt9fNnnoFomd9AhBZGoJYEhhs8EC0NBB3G88bg4IYzhw03nMkxNzhjAHTYYbZivN63b88xfPCg4rC8fPsZswcVFUCHtTdvlsCnhUEggQHJ5SDj8ccKEPAfIKRiFIyCUTAKRjwAAFE8SWaucqFiAAAAAElFTkSuQmCC","orcid":"","institution":"North Bristol NHS Trust, Southmead Hospital","correspondingAuthor":true,"prefix":"","firstName":"Lyn","middleName":"I","lastName":"Jones","suffix":""},{"id":268474580,"identity":"d59d0fc5-1038-4207-a335-949c671256b6","order_by":1,"name":"Andrea Marshall","email":"","orcid":"","institution":"University of Warwick","correspondingAuthor":false,"prefix":"","firstName":"Andrea","middleName":"","lastName":"Marshall","suffix":""},{"id":268474581,"identity":"b7bd8674-8f97-4f18-8c0e-285009ed9c0a","order_by":2,"name":"Rebecca Geach","email":"","orcid":"","institution":"North Bristol NHS Trust, Southmead Hospital","correspondingAuthor":false,"prefix":"","firstName":"Rebecca","middleName":"","lastName":"Geach","suffix":""},{"id":268474582,"identity":"00ed852c-0b03-4ab5-976d-29711d94feee","order_by":3,"name":"Premkumar Elangovan","email":"","orcid":"","institution":"Royal Surrey County Hospital NHS Foundation Trust","correspondingAuthor":false,"prefix":"","firstName":"Premkumar","middleName":"","lastName":"Elangovan","suffix":""},{"id":268474583,"identity":"b6c3e893-22a1-425f-9d72-2c7fc102a64c","order_by":4,"name":"Elizabeth O’Flynn","email":"","orcid":"","institution":"St George’s University Hospitals Foundation Trust","correspondingAuthor":false,"prefix":"","firstName":"Elizabeth","middleName":"","lastName":"O’Flynn","suffix":""},{"id":268474584,"identity":"878c8fb1-e84e-46ec-ad38-7cb405aca053","order_by":5,"name":"Tony Timlin","email":"","orcid":"","institution":"North Bristol NHS Trust, Southmead Hospital","correspondingAuthor":false,"prefix":"","firstName":"Tony","middleName":"","lastName":"Timlin","suffix":""},{"id":268474585,"identity":"d4224d37-c2d7-4f04-b7d5-14b51599f7cf","order_by":6,"name":"Sadie McKeown-Keegan","email":"","orcid":"","institution":"North Bristol NHS Trust, Southmead Hospital","correspondingAuthor":false,"prefix":"","firstName":"Sadie","middleName":"","lastName":"McKeown-Keegan","suffix":""},{"id":268474586,"identity":"43541c60-dd7d-44f0-a7c2-db7898152f48","order_by":7,"name":"Janice Rose","email":"","orcid":"","institution":"Independent Cancer Patients’ Voice","correspondingAuthor":false,"prefix":"","firstName":"Janice","middleName":"","lastName":"Rose","suffix":""},{"id":268474587,"identity":"4969e91f-fce1-4683-8d4b-4c70efa0a79d","order_by":8,"name":"Sarah Vinnicombe","email":"","orcid":"","institution":"Gloucestershire Hospitals NHS Foundation Trust","correspondingAuthor":false,"prefix":"","firstName":"Sarah","middleName":"","lastName":"Vinnicombe","suffix":""},{"id":268474588,"identity":"93d9e0b3-0a03-4059-89da-6543a20269c6","order_by":9,"name":"Sian Taylor-Phillips","email":"","orcid":"","institution":"University of Warwick","correspondingAuthor":false,"prefix":"","firstName":"Sian","middleName":"","lastName":"Taylor-Phillips","suffix":""},{"id":268474589,"identity":"b2d7372c-e890-407b-9db3-404cd56a8353","order_by":10,"name":"Mark Halling-Brown","email":"","orcid":"","institution":"Royal Surrey County Hospital NHS Foundation Trust","correspondingAuthor":false,"prefix":"","firstName":"Mark","middleName":"","lastName":"Halling-Brown","suffix":""},{"id":268474590,"identity":"9e6e6686-7572-400b-aa84-cce2f66240e2","order_by":11,"name":"Janet A Dunn","email":"","orcid":"","institution":"University of Warwick","correspondingAuthor":false,"prefix":"","firstName":"Janet","middleName":"A","lastName":"Dunn","suffix":""}],"badges":[],"createdAt":"2024-01-20 14:19:36","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-3881738/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-3881738/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":50184846,"identity":"50aea681-2555-4fa2-83c5-b1db7a89ccb5","added_by":"auto","created_at":"2024-01-25 20:07:54","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":45229,"visible":true,"origin":"","legend":"\u003cp\u003eFlow diagram detailing participation in FAST MRI interpretation e-learning study\u003c/p\u003e","description":"","filename":"1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-3881738/v1/3338032643463ca2a1083dfe.jpg"},{"id":50184510,"identity":"495336c8-48be-4a3b-9497-08b9b7a48e21","added_by":"auto","created_at":"2024-01-25 19:59:54","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":31885,"visible":true,"origin":"","legend":"\u003cp\u003eDiagnostic accuracy in the receiver operating characteristic (ROC) space\u003c/p\u003e\n\u003ch5\u003ea. Point estimates of accuracy for individual readers in ROC space\u003c/h5\u003e\n\u003ch5\u003eb. Plot of accuracy in ROC space for each group and attendance or non-attendance with error bars for 95%CIs\u003c/h5\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-3881738/v1/7c6616f38ad049dea6548608.png"},{"id":50184512,"identity":"ee0af042-25c7-4e66-b157-84fa49277661","added_by":"auto","created_at":"2024-01-25 19:59:54","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":35459,"visible":true,"origin":"","legend":"\u003cp\u003eChanges in reader specificity with number of test-set FAST MRI scans read over time\u003c/p\u003e\n\u003cp\u003eMulti-level generalised mixed model using restricted cubic splines with 4 knots to the number of scans read over time by attendance or non-attendance at previous FAST-MRI training and by group\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-3881738/v1/f88e9e082bb155ace930505f.png"},{"id":50184511,"identity":"ca523e59-494f-4a4b-bba2-fd1128dd82c9","added_by":"auto","created_at":"2024-01-25 19:59:54","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":28822,"visible":true,"origin":"","legend":"\u003cp\u003eChanges in concordance with the true outcome by number of test-set FAST-MRIs read over time\u003c/p\u003e\n\u003cp\u003eMulti-level generalised mixed model using restricted cubic splines with 4 knots to the number of scans read over time by attendance or non-attendance at previous FAST-MRI training and by group\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-3881738/v1/4ca986b832181bc5b086a158.png"},{"id":50184847,"identity":"b2217227-def7-47db-a5c6-7e8906bc3a80","added_by":"auto","created_at":"2024-01-25 20:07:54","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":14734,"visible":true,"origin":"","legend":"\u003cp\u003eChanges in concordance with the true outcome by number of test-set FAST-MRIs read per batch\u003c/p\u003e\n\u003cp\u003eMulti-level generalised mixed model using restricted cubic splines with 4 knots fitted to the rank order of FAST-MRI scans read per batch by reader group\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-3881738/v1/58c57b14e3561f5939bdd123.png"},{"id":50185178,"identity":"875326f1-f27b-4686-8471-f0efc5ae0e28","added_by":"auto","created_at":"2024-01-25 20:15:55","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":959634,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3881738/v1/e285de5a-5cad-47e3-a561-ccd439476de7.pdf"},{"id":50184849,"identity":"ae427981-85fc-4d2f-9144-f33d1c38241a","added_by":"auto","created_at":"2024-01-25 20:07:54","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":645009,"visible":true,"origin":"","legend":"\u003cp\u003eAdditional File 1 (.pdf): Specification of the FAST MRI protocol and composition of the assessment test-set\u003c/p\u003e\n\u003cp\u003eThe specification of the FAST MRI protocol used in the current study and the composition of the assessment test set used in current study have been previously published. They are reproduced here in line with the copyright policy of the journal in which they were previously published.\u003c/p\u003e","description":"","filename":"Additionalfile1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3881738/v1/2495290e55b62426641d99be.pdf"},{"id":50184514,"identity":"f7258f76-4b31-4635-ad23-19f6475b02b0","added_by":"auto","created_at":"2024-01-25 19:59:54","extension":"pdf","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":58032,"visible":true,"origin":"","legend":"\u003cp\u003eAdditional File 2 (.pdf): Standardised FAST MRI interpretation-training programme\u003c/p\u003e\n\u003cp\u003eThe FAST MRI interpretation-training programme delivered as e-learning in the current study was adapted from a previously developed, standardised, in-person interpretation-training programme described in a previous publication. Details of the training programme have been reproduced here in line with the copyright policy of the journal in which they were previously published.\u003c/p\u003e","description":"","filename":"Additionalfile2.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3881738/v1/6f88c0b2635851b1e57d7575.pdf"},{"id":50184848,"identity":"5684681d-49c7-4a93-b3fc-6a27ddeadc2f","added_by":"auto","created_at":"2024-01-25 20:07:54","extension":"pdf","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":107462,"visible":true,"origin":"","legend":"\u003cp\u003eAdditional File 3 (.pdf): Comparison graphic, included to inform discussion of our conclusions on optimal batch size\u003c/p\u003e\n\u003cp\u003eThe relationship of accuracy with batch size of the subset of readers who completed reading the assessment test set in a single batch of 125 FAST MRI scans is presented (for comparison with Figure 5 - the equivalent graphic for all readers). The information is presented as a graphic entitled: Changes in concordance with the true outcome (accuracy) by scan position within a batch, and by reader group*, showing only the readers that completed reading the assessment test set of 125 FAST MRI scans in a single batch (multi-level generalised mixed model using restricted cubic splines with 4 knots fitted to the rank order of FAST MRI scans read per batch).\u003c/p\u003e","description":"","filename":"Additionalfile3.pdf","url":"https://assets-eu.researchsquare.com/files/rs-3881738/v1/ae3ecf781c9ed6a5c71c0ae2.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Optimising the diagnostic accuracy of First post-contrAst SubtracTed breast MRI (FAST MRI) through interpretation-training: a multicentre e-learning study, mapping the learning curve of NHS Breast Screening Programme (NHSBSP) mammogram readers using an enriched dataset","fulltext":[{"header":"Background","content":"\u003cp\u003eScreening with breast MRI can reduce interval cancers for women with very dense breasts but otherwise at population-risk of breast cancer(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e). First post-contrAst SubtracTed MRI (FAST MRI), a shortened form of breast MRI, retains the sensitivity for aggressive breast cancer of full protocol breast MRI (fpMRI)(\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e), and its shorter acquisition and interpretation times make it more likely to be cost-effective(\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e). The diagnostic accuracy of FAST MRI has been shown to be similar to that of fpMRI when reported by experts in fpMRI interpretation(\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e). FAST MRI is being introduced into clinical practice to screen a wider group of women than those currently screened with fpMRI(\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eInternationally, many fewer radiologists interpret fpMRI than interpret screening mammograms(\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e). Published expert opinion on the implementation of FAST MRI into screening practice has emphasized the importance of performance audit for readers whilst suggesting that benchmarks for interpretation can be developed following roll out without specific interpretation-training for existing fpMRI readers(\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e). However, by excluding mammogram readers who do not currently interpret fpMRI from FAST MRI interpretation, this approach could limit the potential expansion of the role of FAST MRI by limiting the numbers of radiologists who may interpret it. For FAST MRI to be scaled up within screening programmes worldwide, effective FAST MRI interpretation-training for mammogram readers is needed.\u003c/p\u003e \u003cp\u003eThe Society of Breast MRI provided interpretation-training to experienced fpMRI readers prior to their interpretation of FAST MRI within the EA1141 breast screening trial(\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e). No formal evaluation of this training was published but diagnostic accuracy achieved at single read within this trial for FAST MRI was 96% sensitivity and 87% specificity(\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eWe could find only one published multi-centre study evaluating the effectiveness of FAST MRI interpretation-training(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e). This study built on previous work to develop a standardised training programme for NHS Breast Screening Programme (NHSBSP) mammogram readers(\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e). Following a single day’s training, mammogram readers achieved an overall sensitivity of 86% and specificity of 86%. However, the diagnostic accuracy achieved in the study by those with previous experience of reporting fpMRI (sensitivity 89%, specificity 90%) remained higher than for those with no such previous experience (sensitivity 83% p \u0026lt; 0.001, specificity 82% p \u0026lt; 0.001)(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eFormative assessment is an educational technique where the assessment process includes feedback to the learner so that in addition to measuring the learner’s achievement, it enhances learning(\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e). We hypothesised that readers’ diagnostic accuracy could be optimised by converting summative assessment (without any feedback to the readers), as used in previous FAST MRI reader training studies(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) into formative assessment (by giving readers immediate feedback for each FAST MRI scan read during the assessment task).\u003c/p\u003e "},{"header":"Methods","content":"\u003cp\u003e This study was reviewed and approved by the Health and Care Research Wales Ethics Committee and by the Health Research Authority (REC:21/HRA/4543 IRAS:301714) and prospectively registered (ISRCTN:16624917) and all participants gave written informed consent.\u003c/p\u003e\u003cp\u003eThe aims of the study were:\u003c/p\u003e\u003cp\u003e \u003c/p\u003e\u003cul\u003e \u003cli\u003e \u003cp\u003eTo determine whether mammogram readers’ diagnostic accuracy can be improved through the addition of formative assessment(\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e) to standardised FAST MRI interpretation-training\u003c/p\u003e \u003c/li\u003e \u003cli\u003e \u003cp\u003eTo map the learning curve for NHSBSP mammogram readers during FAST MRI interpretation-training (by evaluating the incremental diagnostic accuracy of both experienced and novice MRI readers during the formative assessment process).\u003c/p\u003e \u003c/li\u003e \u003c/ul\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eStudy Design\u003c/p\u003e\u003cp\u003eProspective, blinded interpretation of an enriched dataset by multiple readers.\u003c/p\u003e\u003cp\u003eParticipants and setting\u003c/p\u003e\u003cp\u003eAll NHSBSP mammogram readers who were fully qualified to interpret mammograms, at 7 sites in England were invited to take part (December 2021 – February 2022) and were classified as Group 1 if they also interpreted fpMRI in their normal clinical practice, and Group 2 if not. Both Groups were sub-classified into those who had previously undertaken in-person FAST MRI training as part of a research study (“Attended”) and those that had not (“Not attended”). Participants then independently completed two days of standardised FAST MRI interpretation-training, of which the second day comprised reading a test set of FAST MRI scans with feedback on the true outcome for each scan being given immediately after their opinion was recorded (formative assessment(\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e)). The training was undertaken remotely at times chosen by the readers (January - June 2022).\u003c/p\u003e\u003cp\u003eTest set\u003c/p\u003e\u003cp\u003eThe test set comprised 125 FAST MRI scans with known outcome, acquired as fpMRI during 2015, including a consecutive high-risk screening series (72 scans) enriched with additional cancer cases from the same year (53 scans). All cancer cases had histological confirmation and non-cancer scans were confirmed with two-year follow-up. Details of this test set have been described previously(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) (FAST MRI specification and test set composition have been reproduced in \u003cb\u003eAdditional file 1\u003c/b\u003e). Of 125 FAST MRIs in the test set, 54 had biopsy-confirmed unilateral cancer and one bilateral (56 breasts with cancer) and 2 women had two separate tumours identified in the same breast, giving a total of 58 cancers reported in the ground truth, 56 invasive and 2 ductal carcinoma-in-situ (DCIS).\u003c/p\u003e\u003cp\u003eThe current study used the same test set as the previous interpretation-training studies(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e). The training in the current study differed from that delivered previously in being delivered as remote e-learning and including additional formative assessment. Participants who had taken part in a previous interpretation-training study (11 in Group 1 and 7 in Group 2) were viewing the FAST MRI scans of the test set for the second time. However, prior to the start of the current study, the ground truth (true outcome) of the test set had at no time been revealed to them and the average time interval between reading the test set in one of the two previous studies and in the current study was 24 months (range 17–30 months).\u003c/p\u003e\u003cp\u003eElectronic format\u003c/p\u003e\u003cp\u003ePreviously developed software (RiViewer), that displays FAST MRI as maximum intensity projection (MIP) and stacked, subtracted slices, was used in which biopsy-proven cancers had been drawn onto images electronically as regions of interest (ROI volumes) to provide ground truth. As in a previous study(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e), during hands-on workstation training, learners reviewed 29 training FAST MRI scans, and could discover the ground truth at the touch of a button.\u003c/p\u003e\u003cp\u003eIn the current study the software was adapted so that the test set was presented as formative assessment (providing a second, additional day of training). Once participants had completed their interpretation of each scan and committed their assessment electronically, it was automatically locked in and they were immediately able to view the ground truth of the scan, superimposed on their own opinion. This gave them instant feedback prior to their viewing of the next scan in the test set.\u003c/p\u003e\u003cp\u003eTraining and test set MRIs are mutually exclusive and were from a single centre but acquired during different years, from different women. The readers’ point ROI needed to be within the ground truth ROI volume for their opinion to be registered as correct.\u003c/p\u003e\u003cp\u003eStandardised training\u003c/p\u003e\u003cp\u003eA previously developed standardised training programme(\u003cspan additionalcitationids=\"CR12\" citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e–\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) (details reproduced in \u003cb\u003eAppendix 2\u003c/b\u003e), was adapted to enable remote self-directed independent e-learning by participants. Previously developed small-group presentations and hands-on workstation sessions were recorded and made available to participants online, as videos. Additionally, software was provided to the NHS sites that enabled learners to simultaneously login to the RiViewer on NHS workstations to practice image manipulation of the 29 training FAST MRI scans, guided by the recorded sessions.\u003c/p\u003e\u003cp\u003eReaders were taught how to classify FAST MRI scans according to the UK 5-point breast imaging classification specified for screening fpMRI in women at higher risk of breast cancer within NHSBSP(\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e). When adding a point region of interest (ROI) to an image they were prompted to label the ROI with an MRI classification from the UK 5-point scale. Quantification of the UK 5-point scale, defining how it maps to the BI-RADS classification system was described by Taylor et al.(\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eIn the current study, the new provision of immediate feedback (on the true outcome of each scan) during the test set reading assessment task (termed formative assessment(\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e)) forms a new and additional part of the reader training.\u003c/p\u003e\u003cp\u003eTest set interpretation\u003c/p\u003e\u003cp\u003eHaving completed the training set, participants interpreted the test set of 125 FAST MRIs, blinded to all other information (clinical history, previous imaging, histology, and other readers’ interpretations). Readers were told to expect more cancers than in usual screening practice but no other indication of the number of cancers was given. The test set was presented to each reader in a different random order.\u003c/p\u003e\u003cp\u003eFor the current study, readers were encouraged to complete their reading of the assessment test set (formative assessment task) within as short a time as was reasonably possible, following completion of the other training material. No recommendations were made regarding the number of scans to be read at a time (batch size).\u003c/p\u003e\u003cp\u003eSample size calculation\u003c/p\u003e\u003cp\u003eUsing the results of a previous interpretation-training study(\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e), a sample of 250 breasts from 125 women would allow the lower 95% confidence limit of the inter-rater reliability (Kappa statistic) to be estimated to within 0.07 with a minimum of 6 readers in each group and a proportion of cancers of 0.22(\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e). Thus, to assess inter-rater reliability (we required a minimum total of 12 readers: 6 in each group.\u003c/p\u003e\u003ch2\u003eStatistical analysis\u003c/h2\u003e\u003cp\u003ePer-breast analysis of the frequency of results against true outcome was obtained overall and for each reader. Sensitivity, specificity, and concordance of readers’ FAST MRI classification with the true outcome were determined and differences across reader groups and previous attendance on a FAST MRI training session assessed using a multi-level-generalised-mixed model to account for multiple readers per scan and the dependence between breasts. Restricted cubic splines with 4 knots to the number of FAST MRI scans read overall and per reading session (batch size) were also included in the models to assess whether the readers performance improved during the assessment task.\u003c/p\u003e\u003cp\u003eThe agreement between readers and the true outcome was assessed using Cohen’s κ coefficient, to account for the probability of agreement occurring by chance. The diagnostic odds ratio was determined as a measure of overall diagnostic accuracy independent of prevalence(\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e). Classifications 4 and 5 were considered indicative of cancer, and classifications 1–3 considered a normal result.\u003c/p\u003e\u003cp\u003eInterpretation times were compared across reader groups (Wilcoxon rank-sum).\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eThere were 43 participants from 7 sites, 22 with previous experience of reading fpMRI (Group 1) and 21 new to reading MRI (Group 2). Eighteen participants (11 from Group 1 and 7 from Group 2) had previously undertaken in-person FAST MRI training as part of a research study (\u0026ldquo;Attended\u0026rdquo;) and the remaining participants in each group had not, (\u0026ldquo;Not attended\u0026rdquo;)(11,13). None of the data presented in the results section overlaps with that of the previous two studies.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAll participants completed the training, including reading the formative assessment task (test set) of 125 FAST MRI scans (250 breasts). Individual readers\u0026rsquo; opinions for 4 scans failed to register due to a technical error, giving a total of 10,746 reads.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFigure 1 shows the flow chart of reader recruitment (\u003cstrong\u003eFigure 1\u003c/strong\u003e) and Table 1 details participants\u0026rsquo; professional experience (\u003cstrong\u003eTable 1\u003c/strong\u003e).\u003c/p\u003e\n\u003ch3\u003eTable 1: Demographics of participant mammogram readers\u003c/h3\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eGroup 1*\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eGroup 2*\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eNumber of participants\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e22\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e21\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eProfessional Title**\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003eAdvanced Practitioner\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003eConsultant Radiographer\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003eBreast Clinician\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003eRadiologist Associate Specialist\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003eConsultant Radiologist\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e22\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eProfessional Experience\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003eNumber of years interpreting mammograms: median (range)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e9 (1-22)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e7 (1-28)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003eNumber of mammograms interpreted each year: median (range)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e6000 (3000-12000)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e7000 (4000-10000)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003eParticipant readers who interpret digital breast tomosynthesis (DBT) in normal clinical practice\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e22\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e15\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003eNumber of years interpreting breast MRI: median (range)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e8 (1-25)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003eN/A\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003eNumber of full protocol breast MRI scans interpreted each year: median (range)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e135 (30-450)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003eN/A\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"50.9719222462203%\" valign=\"top\"\u003e\n \u003cp\u003eTotal numbers of participant readers who previously attended in person FAST MRI training\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"22.4622030237581%\" valign=\"top\"\u003e\n \u003cp\u003e11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"26.565874730021598%\" valign=\"top\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e* Reader group: group 1 = mammogram readers with experience of fpMRI interpretation in their usual clinical practice, group 2 = mammogram readers with no previous experience of breast MRI interpretation in their clinical practice.\u003c/p\u003e\n\u003cp\u003e**Professional titles in UK: Screening mammograms within the NHS Breast Screening Programme are interpreted by multidisciplinary healthcare professionals trained in mammogram interpretation. Their performance is subject to continuous audit through the UK Breast Screening Information System that produces individual real-life performance data over rolling 3-year periods (35).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026ldquo;Consultant Radiologist\u0026rdquo; and \u0026ldquo;Breast Clinician\u0026rdquo; are titles held by medical doctors. Consultant Radiologists are registered on the General Medical Council\u0026rsquo;s Specialist Register following Completion of Specialist Training (5 years) with standards and curriculum set by the Royal College of Radiologists (RCR). The Association of Breast Clinicians launched the Credential in Breast Disease Management for Breast Clinicians, jointly with the RCR, in 2019, to standardise and formalise training for Breast Clinicians across the UK (3-year training programme)(36).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u0026ldquo;Advanced Practitioners\u0026rdquo; and \u0026ldquo;Consultant Radiographers\u0026rdquo; are experienced, registered healthcare practitioners, typically mammographers, who have additionally completed specialist training, underpinned by a master\u0026rsquo;s level award or equivalent to support their professional practice within the NHS (37).\u0026nbsp;\u003c/p\u003e\n\u003ch3\u003eFigure 1: Flow diagram detailing participation in FAST MRI interpretation e-learning study\u003c/h3\u003e\n\u003ch3\u003ePer-breast analysis\u003c/h3\u003e\n\u003cp\u003eThe per-breast analysis comparing readers\u0026rsquo; MRI classification with the true outcome (cancer or normal) showed an overall sensitivity of 83% (95%CI 81-84%; 1994/2408) and specificity of 94% (95%CI 93\u0026ndash;94%; 7806/8338).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eReaders with experience of fpMRI interpretation (Group 1) showed similar sensitivity (1034/1232; 84%; 95%CI 82-86%) but slightly higher specificity (4031/4266; 94%; 95%CI 94\u0026ndash;95%) than readers without fpMRI experience (Group 2) (sensitivity = 82%; 95%CI 79\u0026ndash;84% (960/1176) p=0.14; specificity = 93%; 95%CI 92\u0026ndash;93% (3775/4072) p = 0.001) (\u003cstrong\u003eTable 2\u003c/strong\u003e).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThose readers that had previously completed in-person FAST MRI interpretation training (\u0026ldquo;Attended\u0026rdquo;) had a significantly higher overall sensitivity (88%; 95% CI 85-91%) than those that had not attended (80%; 95% CI 78-82%, p\u0026lt;0.0001), but significantly lower specificity (92%; 95% CI 92-93% compared to 94%; 95% CI 94-95%, p=0.003), irrespective of group (\u003cstrong\u003eTable 2\u003c/strong\u003e). The diagnostic accuracy results are summarised in Figure 2, which plots readers\u0026rsquo; accuracy in the receiver operating characteristic space by group and by whether previously attended in person FAST MRI training (\u003cstrong\u003eFigure 2\u003c/strong\u003e).\u003c/p\u003e\n\u003ch3\u003eFigure 2: Diagnostic accuracy in the receiver operating characteristic (ROC) space\u003c/h3\u003e\n\u003col start=\"1\" style=\"list-style-type: lower-alpha;\"\u003e\n \u003cli\u003e\n \u003ch5\u003ePoint estimates of accuracy for individual readers in ROC space\u003c/h5\u003e\n \u003c/li\u003e\n \u003cli\u003e\n \u003ch5\u003ePlot of accuracy in ROC space for each group and attendance or non-attendance with error bars for 95%CIs\u003c/h5\u003e\n \u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eBoth the inter-reader agreement (kappa) of readers with the true outcome and the diagnostic odds ratio (DOR) were higher for Group 1 (kappa 0.77 (95%CI: 0.76-0.80), DOR 89.58 (95% CI 73.26-109.52) than Group 2 (0.73 (0.70-0.75) and 56.49 (46.76-68.25)) and tended to be higher for those participants that had attended previous FAST MRI training (kappa 0.77 (95%CI: 0.75-0.79), DOR 87.95 (95% CI 70.27-110.08) compared to those participants that had not previously completed FAST MRI training (kappa 0.74 (95%CI: 0.72-0.76), DOR 62.98 (95% CI 52.77-75.16) \u0026nbsp;(\u003cstrong\u003eTable 2\u003c/strong\u003e).\u0026nbsp;\u003c/p\u003e\n\u003ch2\u003eTable 2: Readers\u0026rsquo; diagnostic accuracy by group* and by attendance or non-attendance at previous in-person FAST-MRI interpretation-training\u003c/h2\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"13.976705490848586%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eCategory\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"86.02329450915141%\" colspan=\"5\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eMeasure\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.166666666666668%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eConcordance (Accuracy)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eTrue positive rate (Sensitivity)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.833333333333334%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eTrue negative rate (Specificity)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eKappa (95% CI)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"25%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eDiagnostic odds ratio (95% CI)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eAll readers\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.166666666666668%\" valign=\"top\"\u003e\n \u003cp\u003e9800/10746 (91%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16%\" valign=\"top\"\u003e\n \u003cp\u003e1994/2408 (83%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.833333333333334%\" valign=\"top\"\u003e\n \u003cp\u003e7806/8338 (94%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12%\" valign=\"top\"\u003e\n \u003cp\u003e0.75 (0.74-0.77)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"25%\" valign=\"top\"\u003e\n \u003cp\u003e70.67 (61.59-81.09)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"100%\" colspan=\"6\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eReader group*\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eGroup 1*\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.166666666666668%\" valign=\"top\"\u003e\n \u003cp\u003e5065/5498 (92%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16%\" valign=\"top\"\u003e\n \u003cp\u003e1034/1232 (84%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.833333333333334%\" valign=\"top\"\u003e\n \u003cp\u003e4031/4266 (94%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12%\" valign=\"top\"\u003e\n \u003cp\u003e0.77 (0.76-0.80)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"25%\" valign=\"top\"\u003e\n \u003cp\u003e89.58 (73.26-109.52)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eGroup 2*\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.166666666666668%\" valign=\"top\"\u003e\n \u003cp\u003e4735/5248 (90%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16%\" valign=\"top\"\u003e\n \u003cp\u003e960/1176 (82%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.833333333333334%\" valign=\"top\"\u003e\n \u003cp\u003e3775/4072 (93%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12%\" valign=\"top\"\u003e\n \u003cp\u003e0.73 (0.70-0.75)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"25%\" valign=\"top\"\u003e\n \u003cp\u003e56.49 (46.76-68.25)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"100%\" colspan=\"6\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eAttendance at previous in person FAST MRI interpretation training\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eAttended\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.166666666666668%\" valign=\"top\"\u003e\n \u003cp\u003e4123/4500 (92%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16%\" valign=\"top\"\u003e\n \u003cp\u003e876/1008 (87%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.833333333333334%\" valign=\"top\"\u003e\n \u003cp\u003e3247/3492 (93%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12%\" valign=\"top\"\u003e\n \u003cp\u003e0.77 (0.75-0.79)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"25%\" valign=\"top\"\u003e\n \u003cp\u003e87.95 (70.27-110.08)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eNot attended\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.166666666666668%\" valign=\"top\"\u003e\n \u003cp\u003e5677/6246 (91%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16%\" valign=\"top\"\u003e\n \u003cp\u003e1118/1400 (80%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.833333333333334%\" valign=\"top\"\u003e\n \u003cp\u003e4559/4846 (94%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12%\" valign=\"top\"\u003e\n \u003cp\u003e0.74 (0.72-0.76)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"25%\" valign=\"top\"\u003e\n \u003cp\u003e62.98 (52.77-75.16)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"100%\" colspan=\"6\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eReader group* and attendance at previous in person FAST MRI interpretation training\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eGroup 1*\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003eAttended\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.166666666666668%\" valign=\"top\"\u003e\n \u003cp\u003e1613/1750 (92%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16%\" valign=\"top\"\u003e\n \u003cp\u003e533/616 (87%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.833333333333334%\" valign=\"top\"\u003e\n \u003cp\u003e2002/2134 (94%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12%\" valign=\"top\"\u003e\n \u003cp\u003e0.78 (0.75-0.81)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"25%\" valign=\"top\"\u003e\n \u003cp\u003e97.40 (72.83-130.25)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eGroup 1* Not a\u003c/strong\u003e\u003cstrong\u003ettended\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.166666666666668%\" valign=\"top\"\u003e\n \u003cp\u003e2530/2748 (92%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16%\" valign=\"top\"\u003e\n \u003cp\u003e501/616 (81%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.833333333333334%\" valign=\"top\"\u003e\n \u003cp\u003e2029/2132 (95%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12%\" valign=\"top\"\u003e\n \u003cp\u003e0.77 (0.74-0.80)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"25%\" valign=\"top\"\u003e\n \u003cp\u003e85.82 (64.65-113.93)\u003c/p\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eGroup 2*\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003eAttended\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.166666666666668%\" valign=\"top\"\u003e\n \u003cp\u003e1588/1750 (91%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16%\" valign=\"top\"\u003e\n \u003cp\u003e343/392 (88%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.833333333333334%\" valign=\"top\"\u003e\n \u003cp\u003e1245/1358 (92%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12%\" valign=\"top\"\u003e\n \u003cp\u003e0.75 (0.71-0.78)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"25%\" valign=\"top\"\u003e\n \u003cp\u003e77.12 (54.03-110.09)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"14%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eGroup 2* Not a\u003c/strong\u003e\u003cstrong\u003ettended\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.166666666666668%\" valign=\"top\"\u003e\n \u003cp\u003e3147/3498 (90%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"16%\" valign=\"top\"\u003e\n \u003cp\u003e617/784 (79%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.833333333333334%\" valign=\"top\"\u003e\n \u003cp\u003e2530/2714 (93%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"12%\" valign=\"top\"\u003e\n \u003cp\u003e0.71 (0.69-0.74)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"25%\" valign=\"top\"\u003e\n \u003col\u003e\n \u003cli\u003e(40.48-63.76)\u003c/li\u003e\n \u003c/ol\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e* Reader group: group 1 = experience of fpMRI interpretation in their usual clinical practice, group 2 = no previous experience of breast MRI interpretation in their clinical practice\u003c/p\u003e\n\u003ch3\u003ePlotting the Learning Curve\u003c/h3\u003e\n\u003cp\u003eReaders\u0026rsquo; sensitivity remained fairly stable during the test set reading process (formative assessment task) (p=0.24) and this effect was similar for both groups (interaction p=0.30) and whether or not they had previously completed FAST MRI training (interaction p=0.97).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eHowever, specificity was significantly affected by the number of scans read in the formative assessment task (p\u0026lt;0.001) and this effect differed, depending on whether readers had attended previous FAST MRI training or not (interaction p=0.01) but not between groups (interaction p=0.08). The predicted specificity curves for readers that had attended previous FAST MRI training reached a peak after 75 reads but continued to increase for those that had not attended previous FAST MRI training, with group 1 readers having significantly higher specificity than group 2 (p=0.003) (\u003cstrong\u003eFigure 3\u003c/strong\u003e). \u0026nbsp;\u0026nbsp;\u003c/p\u003e\n\u003ch3\u003eFigure 3: Changes in reader specificity with number of test-set FAST MRI scans read over time\u0026nbsp;\u003c/h3\u003e\n\u003cp\u003eMulti-level generalised mixed model using restricted cubic splines with 4 knots to the number of scans read over time by attendance or non-attendance at previous FAST-MRI training and by group\u003c/p\u003e\n\u003cp\u003eAccuracy (concordance with the true outcome) changed significantly by the number of FAST MRI scans read in the test set reading process (formative assessment task) (p=0.002) and differed depending on whether or not a reader had previously attended FAST MRI training (interaction p=0.02) but was similar for both groups (interaction p=0.36). Accuracy was significantly higher for Group 1 than Group 2 overall (p=0.001) and reached a peak after 75 reads for those readers that had previously attended FAST MRI training, as seen with the results for specificity (\u003cstrong\u003eFigure 4\u003c/strong\u003e).\u003c/p\u003e\n\u003ch3\u003eFigure 4: Changes in concordance with the true outcome by number of test-set FAST-MRIs read over time\u003c/h3\u003e\n\u003cp\u003eMulti-level generalised mixed model using restricted cubic splines with 4 knots to the number of scans read over time by attendance or non-attendance at previous FAST-MRI training and by group\u003c/p\u003e\n\u003ch3\u003eReading pattern: batch size\u0026nbsp;\u003c/h3\u003e\n\u003cp\u003eReaders read the formative assessment task of 125 FAST MRI scans in a median of 2 batches (range 1- 8 batches) with a median of 32 scans read per batch (range 1-125 scans/batch). The readers that did not attend the previous FAST MRI in person training tended to complete this current training in fewer batches and hence had larger batch sizes than those that had attended previous training (\u003cstrong\u003eTable 3\u003c/strong\u003e). \u0026nbsp;\u003c/p\u003e\n\u003ch2\u003eTable 3: Batch reading pattern by group* and by attendance or non-attendance at previous in-person FAST-MRI interpretation-training\u003c/h2\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"601\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eGroup 1*\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eGroup 1*\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.666666666666666%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eGroup 2*\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eGroup 2*\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eAttended\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eNot attended\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.666666666666666%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eAttended\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eNot attended\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eNumber of batches\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.666666666666666%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eMedian (IQR)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e2 (2-4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e2 (1-4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.666666666666666%\" valign=\"top\"\u003e\n \u003cp\u003e3 (3-5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e1 (1-3)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eRange\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e1-8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e1-5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.666666666666666%\" valign=\"top\"\u003e\n \u003cp\u003e2-8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e1-7\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eNumber of FAST MRI scans read in a batch\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.666666666666666%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eMedian (IQR)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e31 (15-58)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e33 (23-63)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.666666666666666%\" valign=\"top\"\u003e\n \u003cp\u003e30 (14-41)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e50 (19-125)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"29.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eRange\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"17.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e9-125\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.833333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e7-125\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"15.666666666666666%\" valign=\"top\"\u003e\n \u003cp\u003e1-84\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"18.333333333333332%\" valign=\"top\"\u003e\n \u003cp\u003e1-125\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e* Reader group: group 1 = experience of fpMRI interpretation in their usual clinical practice, group 2 = no previous experience of breast MRI interpretation in their clinical practice\u003c/p\u003e\n\u003cp\u003eAccuracy (concordance with the true outcome) significantly changed depending on the number of reads within a batch (p=0.02) but in a similar manner for both groups (interaction p=0.53) and for\u0026nbsp;whether or not a reader had previously attended FAST MRI training (interaction p=0.78). Accuracy\u0026nbsp;tended to worsen after 50 FAST MRI scans were read within a batch for both groups (\u003cstrong\u003eFigure 5\u003c/strong\u003e).\u0026nbsp;\u003c/p\u003e\n\u003ch3\u003eFigure 5: Changes in concordance with the true outcome by number of test-set FAST-MRIs read per batch\u003c/h3\u003e\n\u003cp\u003eMulti-level generalised mixed model using restricted cubic splines with 4 knots fitted to the rank order of FAST-MRI scans read per batch by reader group\u003c/p\u003e\n\u003cp\u003eSimilarly, specificity significantly changed with the number of reads within a batch (p= 0.0001) for both groups (interaction p=0.18). Sensitivity, although not significantly dependent, tended to worsen with increasing number of MRI scans read within a batch (p= 0.08) and this was similar for both groups (interaction p=0.91). \u0026nbsp;\u003c/p\u003e\n\u003ch3\u003eTime taken to interpret\u003c/h3\u003e\n\u003cp\u003eThe median time taken for the individual readers to interpret each FAST MRI scan was 22 seconds less for Group 1 (median 56 seconds, range 8-47466 seconds) than for Group 2 (median 78 seconds, 14-22830 seconds, p \u0026lt;0.0001). \u0026nbsp;Seven records had a total time of more than 1000 seconds.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eIn comparison with previous FAST MRI training results from the two studies that used in-person versions of the same standardised training programme and assessment dataset (delivered as one-to-one(\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) or small group(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e) training), FAST MRI readers in the current study (trained with remote e-learning and automated formative assessment) achieved higher specificity (94%; 95%CI 93\u0026ndash;94 vs. 87%; 85\u0026ndash;89 (one-to-one training)(\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) and 86%; 85\u0026ndash;86 (small group training)(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e)), and higher overall diagnostic accuracy (reader agreement with the true outcome)(kappa 0.75; 0.74\u0026ndash;0.77 vs. 0.69; 0.65\u0026ndash;0.72(\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) and 0.63; 0.61\u0026ndash;0.65(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e), and DOR 70.67 (61.59\u0026ndash;81.09) vs. 48.30 (35.12\u0026ndash;66.44)(\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) and 35.49 (30.87\u0026ndash;40.81)(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e)). There was, however, a trend for lower sensitivity (83%; 81\u0026ndash;84 vs. 88%; 84\u0026ndash;91(\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e) and 86% 84\u0026ndash;87(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e)) at cancer detection.\u003c/p\u003e \u003cp\u003eSpecificity improved during the test set interpretation assessment task, similarly for both groups but the effect differed depending on whether readers had attended previous training sessions. Whilst readers that had attended previous FAST MRI training tended to reach a peak after 75 reads, the specificity continued to improve for those that had not attended previous FAST MRI training. Similar learning curves were also seen for concordance with the true outcome. Sensitivity remained stable over all reads.\u003c/p\u003e \u003cp\u003eSpecificity is arguably the most important diagnostic accuracy parameter to optimise in the context of a breast screening tool that is designed for women with a low pre-test probability. This is because small changes in specificity can have a large effect on the number of false positive recalls in a population screening programme, with each recall causing harm to the woman screened and also incurring a financial and workforce cost(\u003cspan additionalcitationids=\"CR15\" citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e). The specificity for mammography in the NHS Breast Screening Programme (NHSBSP), interpreted by double-reading, is 96%(\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e). The specificity achieved at single read of an enriched test set of FAST MRI scans by readers in the current study could be considered comparable (94%).\u003c/p\u003e \u003cp\u003eThese results demonstrate that the inclusion of immediate feedback for each scan during test set interpretation in FAST MRI reader training optimised specificity whilst maintaining high levels of sensitivity, which would suit a screened population with low pre-test probability.\u003c/p\u003e \u003cp\u003eAchievement of reporting benchmarks for fpMRI\u003c/p\u003e \u003cp\u003eTwo days of standardised FAST MRI interpretation-training, undertaken as remote e-learning, enabled NHSBSP mammogram-readers, both those experienced in fpMRI interpretation (Group 1) and novice MRI readers (Group 2), to achieve, at single read of an enriched dataset, benchmarks set for fpMRI interpretation in practice by the American College of Radiology\u0026rsquo;s Breast Imaging Reporting and Data System (BI-RADS) for both sensitivity (Groups 1 (84%) and 2 (82%) vs. \u0026gt;80% BI-RADS benchmark(\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e)) and specificity (Groups 1 (94%) and 2 (93%) vs. \u0026gt;85% BI-RADS benchmark(\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e)). Of 43 participants, the two-day remote e-learning programme was sufficient for 43/43 (100%) to achieve specificity above the 85% BI-RADS benchmark and for 33/43 (77%) to achieve sensitivity above the 80% BI-RADS benchmark.\u003c/p\u003e \u003cp\u003eNovice MRI readers (Group 2) achieved similar sensitivity to experienced fpMRI readers (Group 1) (p\u0026thinsp;=\u0026thinsp;0.14) but lower specificity (p\u0026thinsp;=\u0026thinsp;0.001) although specificity differed between groups by only one percentage point (Group 1: 94% and Group 2: 93%).\u003c/p\u003e \u003cp\u003eComparison between the performance of those who had previously attended in-person FAST MRI interpretation training and those who had not\u003c/p\u003e \u003cp\u003eWhilst the reader agreement with the true outcome (kappa) and the DOR did not differ significantly between the readers who had previously attended in-person FAST MRI interpretation-training (11/22 in Group 1 and 7/21 in Group 2) and those who had not, the sensitivity for cancer detection was higher and the specificity lower for the \u0026ldquo;attended\u0026rdquo; cohort than for the \u0026ldquo;not attended\u0026rdquo; cohort. Looking at the individual performance, during a previous study(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e), of the 14 participants of the current study who had attended previous small group training, 8 of these participants had a sensitivity in the top 9 sensitivities of participants in the previous study and none were in the bottom 7 sensitivities(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e). Additionally, 8 of these participants had specificity in the bottom 12 for specificity in the previous study and 3 were in the top 11 specificities(\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e). Therefore, self-selection bias could have contributed to the within group significant differences of sensitivity and specificity found for attendance vs. non-attendance at previous in person training.\u003c/p\u003e \u003cp\u003eLiterature comparison \u0026ndash; the effect of batch size on diagnostic performance\u003c/p\u003e \u003cp\u003eThe Co-Ops Study assessed the effect of reading practice, including batch size, on reader diagnostic performance in mammography within the NHSBSP and demonstrated increased specificity with increased batch size up to 40 mammograms per batch with the trend continuing in longer batches(\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e). The current study, whilst it showed a trend for increasing specificity with batch size up to 50 FAST MRI scans per batch and decreasing sensitivity with increasing batch size, also demonstrated that concordance with the true outcome (as a measure of overall accuracy) tended to worsen when more than 50 scans were read within one batch. This accords with results from a study of 2,937,312 mammogram reads that demonstrated both small increases in specificity and small decreases in sensitivity for mammograms read at later positions within a batch. The authors of the study suggested that optimal batch-size for reading mammograms could be 60\u0026ndash;70 reads per batch(\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eOne possible explanation for the optimal batch size for FAST MRI (50 scans per batch) being smaller than that suggested for mammograms could be the difference in complexity between reading FAST MRI scans and mammograms. Reading FAST MRI scans in the current study could more quickly cause fatigue for readers than reading mammograms because FAST MRI reading format requires more images to be reviewed per scan than for a mammogram. However, the reading format of digital breast tomosynthesis (DBT)(2D plus stack of reconstructed slabs) has a similar complexity to that of FAST MRI (MIP plus stack of slices) and although we could find no study that reported the effect of reading batch size on the diagnostic accuracy of DBT, evidence of increasing reader fatigue during the process of reading a batch of 40 DBT scans has been reported(\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eLiterature comparison \u0026ndash; reading times\u003c/p\u003e \u003cp\u003eThe reading times achieved by readers in this study (56 and 78 seconds for Groups 1\u0026amp;2) were longer than times reported for NHSBSP mammogram readers to interpret mammograms (35 and 76 seconds (\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e, \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e)) and about half that reported for NHSBSP mammogram readers to interpret DBT (2.81 minutes)(\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e). However, evidence is emerging that various AI strategies may reduce reading times for DBT without affecting accuracy(\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e, \u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e). In the future similar approaches may prove valuable for FAST MRI.\u003c/p\u003e \u003cp\u003eLimitations of the current study\u003c/p\u003e \u003cp\u003eReaders who had previously attended FAST MRI interpretation training had interpreted the same test set of 125 FAST MRI scans during the previous study. However, since they had not previously seen the ground truth (true outcome) of the scans in the test set at any time, and there was an average time interval of 24 months (range 17\u0026ndash;30 months) between reading the test set in the two studies, it is unlikely that their diagnostic performance was affected by this.\u003c/p\u003e \u003cp\u003eThe test set was read outside normal clinical practice and therefore reader performance is likely to have been subject to a laboratory effect(\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eReaders were free to self-select batch length when reading the test set assessment task. Therefore, our conclusions on optimal batch size could potentially have been confounded through self-selection bias. However, similar results were seen with the subset of readers who completed all 125 scans of the test set in a single batch (7 from Group 1 and 8 from Group 2) (\u003cb\u003eAppendix 3\u003c/b\u003e), suggesting the effect of self-selection bias, although unquantifiable, is likely to be small.\u003c/p\u003e \u003cp\u003eImplications of the research\u003c/p\u003e \u003cp\u003eThe single reading performance at FAST MRI achieved by experienced (Group 1) and novice (Group 2) readers in the current study, reading an enriched dataset, compares well with published figures for diagnostic performance at fpMRI for radiologists experienced in breast MRI interpretation in community screening practice in the USA (Breast Cancer Surveillance Consortium (BCSC)(\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e): sensitivity: 84% (Group 1) and 82% (Group 2) vs. 81% (BCSC), and specificity: 94% (Group 1) and 93% (Group 2) vs. 83% (BCSC).\u003c/p\u003e \u003cp\u003eDouble reading is the current standard for reading within NHSBSP and has the potential to further improve diagnostic accuracy for mammogram readers who have undertaken standardised FAST MRI interpretation training.\u003c/p\u003e \u003cp\u003eFAST MRI was designed as a screening test that would provide the high sensitivity of fpMRI for aggressive breast cancers at a fraction of the cost through shorter acquisition and reading times(\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e), with the intention that it could be used to screen a wider population than currently benefit from screening with fpMRI(\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e, \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e). However, for mass screening tests, specificity is arguably the most important metric(\u003cspan additionalcitationids=\"CR15\" citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e) and trials of breast MRI (scans single read by expert fpMRI readers) for women with dense breasts, but otherwise at population risk of breast cancer, have reported results with lower specificity (86.7%(\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e) and 92.6%(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e)) than sensitivity (95.7%(\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e) and 95.2%(\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e)). The specificity achieved for FAST MRI by mammogram readers in the current study following 2 days of standardised training (94%) compares well with the results from both these MRI screening trials and approaches the specificity of mammography achieved with double reading within the NHSBSP for population screening (96%)(\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e).\u003c/p\u003e"},{"header":"Conclusions","content":"\u003cp\u003eFuture trials of FAST MRI will benefit from standardising the training, assessment, and credentialing of FAST MRI readers.\u003c/p\u003e \u003cp\u003eProviding formative assessment as part of interpretation-training and optimising reading batch size can increase specificity and provide high diagnostic accuracy at FAST MRI.\u003c/p\u003e \u003cp\u003eThe diagnostic accuracy achieved at single read by NHSBSP mammogram readers in this study suggests that two-day standardised FAST MRI remote e-learning, that includes formative assessment using an enriched dataset, could form the basis for FAST MRI interpretation-training for mammogram readers who wish to participate as readers in future FAST MRI trials and clinical practice. Credentialling readers could be accomplished using BI-RADS benchmarks of performance(\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e) as cut offs for sensitivity and specificity achieved by readers in the assessment task.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eACR\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eAmerican College of Radiology\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eANOVA\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eanalysis of variance\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eAUC\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eArea under a curve\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eBCSC\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eBreast Cancer Screening Consortium\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eBSIS\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eBreast Screening Information Service\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eBI-RADS\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eAmerican College of Radiology\u0026rsquo;s Breast Imaging Reporting and Data System\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eCI\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eConfidence interval\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eDCIS\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eDuctal carcinoma in-situ\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eDOR\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eDiagnostic Odds Ratio\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eFAST MRI\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eFirst post contrast subtracted images (abbreviated breast MRI)\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003efpMRI\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eFull protocol breast Magnetic Resonance Imaging\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eIRAS\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eIntegrated Research Application System for applications to the Health Research Authority and the Research Ethics Committee\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eISRCTN\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eInternational Standard Randomised Controlled Trial Number (however, over the years the scope of the registry has widened beyond randomised controlled trials to include any study designed to assess the efficacy of health interventions in a human population.)\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eMIP\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eMaximum intensity projection image\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eMRI\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eMagnetic Resonance Imaging\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eNHSBSP\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eNational Health Service (United Kingdom) Breast Screening Programme\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003ePERFORMS\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003ePersonal Performance in Mammographic Screening\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eREC\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eResearch Ethics Committee\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eROI\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eregion of interest\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eUK\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eUnited Kingdom of Great Britain and Northern Ireland\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003e\u003cb\u003eUSA\u003c/b\u003e\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eUnited States of America\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cu\u003eEthics approval and consent to participate\u003c/u\u003e: In accordance with the Declaration of Helsinki on research involving human participants, this study was reviewed and approved by the Health and Care Research Wales Ethics Committee REC:21/HRA/4543 and by the Health Research Authority (IRAS:301714).[LC1] \u0026nbsp;The study was prospectively registered (ISRCTN:16624917).\u003c/p\u003e\n\u003cp\u003eAll participants gave informed consent (written) to their participation in the study.\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eConsent for publication\u003c/u\u003e\u003cstrong\u003e:\u003c/strong\u003e Consent to participate in this study included consent for publication of non-identifiable data and was given by all participants.\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eAvailability of data and materials:\u003c/u\u003e The dataset generated and analysed during the current study is not yet publicly available because it is currently being developed into a publicly shareable format. Instead, it is available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eCompeting interests:\u003c/u\u003e Other than the funding sources declared below, the authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eFunding:\u003c/u\u003e This manuscript presents independent research funded by Health Education England through a Bursary for Educational Innovation from the National Breast Imaging Academy (Mapping the learning curve of NHS Breast Screening Programme (NHSBSP) mammogram readers new to the interpretation of a shortened magnetic resonance imaging scan (MRI) of the breast called FAST MRI to support the delivery of a future multicentre trial of FAST MRI versus mammogram for breast cancer screening).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe electronic standardised teaching and assessment tools that were used within the new FAST MRI e-learning programme described in this manuscript were originally developed during work funded by the National Institute for Health Research (Research for Patient Benefit (RfPB), Refinement and piloting of a training programme within the NHS Breast Screening Programme (NHSBSP) workforce of image readers to enable standardised interpretation of a shortened magnetic resonance imaging scan (MRI) of the breast called FAST MRI to support the delivery of a future multicentre trial of FAST MRI versus mammogram for breast cancer screening, PB-PG-1217-20008)(11). The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research (NIHR) or the Department of Health and Social Care.\u003c/p\u003e\n\u003cp\u003eAuthor STP is supported by an NIHR Career Development Fellowship (CDF \u0026ndash; 2016-09-018). The views expressed in this manuscript are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care.\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eAuthors\u0026rsquo; contributions:\u003c/u\u003e LJ, AM and JD contributed substantially to the conception of the work. LJ, JD, EOF, RG, AM, STP, PE and SV had substantial input to the study design. PE, MHB, LJ and RG contributed substantially to the creation of new software used in the study. TT, PE, AM, LJ, RG and SMK had substantial input to the acquisition of data, while AM and PE conducted the data analysis and LJ, AM, STP, SV, EOF and JD contributed substantially to the data interpretation. LJ, AM and TT drafted the work and subsequently, with additional help from STP, SV, RG, and EOF, substantially revised it. The corresponding author is LJ.\u003c/p\u003e\n\u003cp\u003eAll authors have approved the submitted version of this manuscript and have agreed both to be personally accountable for the author\u0026rsquo;s own contributions and to ensure that questions related to the accuracy or integrity of any part of the work, including ones in which the author was not personally involved, are appropriately investigated, resolved, and the resolution documented in the literature. \u0026nbsp;\u003c/p\u003e\n\u003ch3\u003eAcknowledgements\u003c/h3\u003e\n\u003cp\u003eThis study was performed on behalf of the FAST MRI Study Group [LC2] which at the time of this study, in addition to the authors comprised: Clare Alison, Karen Atkinson, Miklos Barta, Gemini Beckett, Claudia Betancourt, Julie Bramwell, Holly Brown, Helen Burt, Louise Cann, Nick Carter, Claire Cartledge, Jane Ceney, Gillian Clark, Eleanor Cornford, Elizabeth Cullimore, Si\u0026acirc;n Curtis, Diana Dalgliesh, Jonathon Delve, Sarah Doyle, Alison Duncan, Holly Elbert, Sarah Fearn, Christopher Foy, Zsolt Friedrich, Hesam Ghiasvand, John Gifford, Dagmar Godden, Zoe Goldthorpe, Sandra Gomes, Narayan Aradhana Goud, Rosie Gray, Sam A. Harding, Kristin Henning, Lucinda Hobson, Claire Hulme, Paula Hynam, El Sanharawi Imane, Emma Jackson, Asif Jaffa, Ragini Jhalla, Margaret Jenkin, Thomas William Jones, Nahid Kamangari, Vandana Kaur, Beckie Kingsnorth, Katherine Klimczak, Elisabeth Kutt, Karen Litton, Simon Lloyd, Iain Lyburn, Anjum Mahatma, Anna Mankelow, Helen Massey, Helen Matthews, Karis McFeely, Clare McLachlan, Sarah McWilliams, Shahrooz Mohammadi, Alice Moody, Elizabeth Muscat, Sreenivas Muthyala, Sarah Perrin, Alison Peters, Alice Pocklington, Elizabeth Preston, Jasvinder Rai, Jo Robson, Corri Salter, Toni Scanlon, Anuma Shrestha, Richard Sidebottom, Mary Sinclair, Sravya Singamaneni, Jim Steel, Lesley Stephenson, Sam Stewart-Maggs, Cheryl Stubbs, Michelle Taylor, Victoria Taylor, Olivia Taylor-Fry, Erika Toth, Matthew Trumble, Alexandra Valencia, Frances Vincent, Anna Wang, Lucy Warren, Sharon Watkin, Sue Widdison, Jennifer Williams and Jennifer Wookey.\u003c/p\u003e\n\u003cp\u003eThe authors wish to thank the Breast Unit Support Trust (BUST) and the Independent Cancer Patients\u0026rsquo; Voice (ICPV) charities and the NIHR Research Design Service (RDS) for their invaluable support.\u003c/p\u003e\n\u003cdiv id=\"_com_2\" language=\"JavaScript\"\u003e\u003cbr\u003e\u003c/div\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eBakker MF, De Lange S V., Pijnappel RM, Mann RM, Peeters PHM, Monninkhof EM, et al. Supplemental MRI screening for women with extremely dense breast tissue. New England Journal of Medicine [Internet]. 2019;381(22):2091\u0026ndash;102. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.nejm.org/doi/full/\u003c/span\u003e\u003cspan address=\"https://www.nejm.org/doi/full/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1056/NEJMoa1903986\u003c/span\u003e\u003cspan address=\"10.1056/NEJMoa1903986\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVeenhuizen SG, De Lange S V., Bakker MF, Pijnappel RM, Mann RM, Monninkhof EM, et al. Supplemental Breast MRI for Women with Extremely Dense Breasts: Results of the Second Screening Round of the DENSE Trial. Radiology [Internet]. 2021;00:1\u0026ndash;9. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1148/radiol.2021203633\u003c/span\u003e\u003cspan address=\"10.1148/radiol.2021203633\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eComstock CE, Gatsonis C, Newstead GM, Snyder BS, Gareen IF, Bergin JT, et al. Comparison of Abbreviated Breast MRI vs Digital Breast Tomosynthesis for Breast Cancer Detection Among Women With Dense Breasts Undergoing Screening. JAMA - Journal of the American Medical Association [Internet]. 2020;323(8):746\u0026ndash;56. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://jamanetwork.com/journals/jama/fullarticle/2761645\u003c/span\u003e\u003cspan address=\"https://jamanetwork.com/journals/jama/fullarticle/2761645\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKuhl CK, Schrading S, Strobel K, Schild HH, Hilgers RD, Bieling HB. Abbreviated breast Magnetic Resonance Imaging (MRI): First postcontrast subtracted images and maximum-intensity projection - A novel approach to breast cancer screening with MRI. Journal of Clinical Oncology [Internet]. 2014;32:2304\u0026ndash;10. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://ascopubs.org/doi/10.1200/JCO.2013.52.5386\u003c/span\u003e\u003cspan address=\"https://ascopubs.doi/10.1200/JCO.2013.52.5386\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVinnicombe S, Harvey H, Healy N. Introduction of an abbreviated breast MRI service in the UK as part of the BRAID trial: practicalities, challenges and future directions. Clin Radiol [Internet]. 2021;76(6):427\u0026ndash;33. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.crad.2021.01.020\u003c/span\u003e\u003cspan address=\"10.1016/j.crad.2021.01.020\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGeach R, Jones LI, Harding SA, Marshall A, Taylor-Phillips S, Mckeown-keegan S, et al. The potential utility of abbreviated breast MRI (FAST MRI) as a tool for breast cancer screening: a systematic review and meta-analysis. Clin Radiol [Internet]. 2021;76:154.e11-154.e22. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.crad.2020.08.032\u003c/span\u003e\u003cspan address=\"10.1016/j.crad.2020.08.032\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBaxter GC, Selamoglu A, Mackay JW, Bond S, Gray E, Gilbert FJ. A meta-analysis comparing the diagnostic performance of abbreviated MRI and a full diagnostic protocol in breast cancer. Clin Radiol [Internet]. 2021;76:154.e23-154.e32. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.crad.2020.08.036\u003c/span\u003e\u003cspan address=\"10.1016/j.crad.2020.08.036\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGrimm LJ, Mango VL, Harvey JA, Plecha DM, Conant EF. Implementation of Abbreviated Breast MRI for Screening: AJR Expert Panel Narrative Review. American Journal of Roentgenology [Internet]. 2022;218(2):202\u0026ndash;12. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.2214/AJR.21.26349\u003c/span\u003e\u003cspan address=\"10.2214/AJR.21.26349\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMarshall HN, Plecha DM. Setting up an Abbreviated Breast MRI Program: Our Two-year Implementation Experience. J Breast Imaging [Internet]. 2020;2(6):603\u0026ndash;8. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1093/jbi/wbaa060\u003c/span\u003e\u003cspan address=\"10.1093/jbi/wbaa060\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee CS, Moy L, Hughes D, Golden D, Bhargavan-Chatfield M, Hemingway J, et al. Radiologist characteristics associated with interpretive performance of screening mammography: A national mammography database (nmd) study. Radiology [Internet]. 2021 Sep 1 [cited 2023 Jun 12];300(3):518\u0026ndash;28. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1148/radiol.2021204379\u003c/span\u003e\u003cspan address=\"10.1148/radiol.2021204379\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJones LI, Marshall A, Elangovan P, Geach R, McKeown-Keegan S, Vinnicombe S, et al. Evaluating the effectiveness of abbreviated breast MRI (abMRI) interpretation training for mammogram readers: a multi-centre study assessing diagnostic performance, using an enriched dataset. Breast Cancer Research [Internet]. 2022;24(1). Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1186/s13058-022-01549-5\u003c/span\u003e\u003cspan address=\"10.1186/s13058-022-01549-5\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHarding S, Geach R, Jones L. The use of \u0026lsquo;Think-Out-Loud\u0026rsquo; methodology in the development of teaching materials for abbreviated breast Magnetic Resonance Imaging scan (FAST MRI) interpretation, and a comparison of the learning experience of two reader cohorts. Eur J Radiol Open [Internet]. 2019;6(May):220\u0026ndash;4. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.ejro.2019.06.002\u003c/span\u003e\u003cspan address=\"10.1016/j.ejro.2019.06.002\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJones LI, Geach R, Harding SA, Foy C, Taylor V, Marshall A, et al. Can mammogram readers swiftly and effectively learn to interpret first post-contrast acquisition subtracted (FAST) MRI, a type of abbreviated breast MRI?: a single centre data-interpretation study. Br J Radiol [Internet]. 2019;92(July):20190663. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1259/bjr.20190663\u003c/span\u003e\u003cspan address=\"10.1259/bjr.20190663\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTaylor-Phillips S, Seedat F, Kijauskaite G, Marshall J, Halligan S, Hyde C, et al. UK National Screening Committee\u0026rsquo;s approach to reviewing evidence on artificial intelligence in breast cancer screening. Lancet Digit Health [Internet]. 2022;4(7):e558\u0026ndash;65. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/S2589-7500(22)00088-7\u003c/span\u003e\u003cspan address=\"10.1016/S2589-7500(22)00088-7\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCole P MA. Basic issues in population screening for cancer. J Natl Cancer Inst. 1980;64(5):1263\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFerrante Di Ruffano L, Hyde CJ, McCaffery KJ, Bossuyt PMM, Deeks JJ. Assessing the value of diagnostic tests: A framework for designing and evaluating trials. BMJ (Online) [Internet]. 2012;344(7847). Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1136/bmj.e686\u003c/span\u003e\u003cspan address=\"10.1136/bmj.e686\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBurnside ES, Vulkan MSD, Blanks RG, Duffy SW. Association between Screening Mammography Recall Rate and Interval Cancers in the UK Breast Cancer Service Screening Program: A Cohort Study. Radiology [Internet]. 2018;288:47\u0026ndash;54. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1148/radiol.2018171539\u003c/span\u003e\u003cspan address=\"10.1148/radiol.2018171539\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBlack P, Wiliam D. \u0026ldquo;In Praise of Educational Research\u0026rdquo;: Formative assessment. Br Educ Res J [Internet]. 2003;29(5):623\u0026ndash;37. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/0141192032000133721\u003c/span\u003e\u003cspan address=\"10.1080/0141192032000133721\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePublic Health England. Technical guidelines for magnetic resonance imaging (MRI) for the surveillance of women at higher risk of developing breast cancer (NHSBSP Publication No 68) [Internet]. Gov.Uk. 2012. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.gov.uk/government/publications/nhs-breast-screening-using-mri-with-higher-risk-women\u003c/span\u003e\u003cspan address=\"https://www.gov.uk/government/publications/nhs-breast-screening-using-mri-with-higher-risk-women\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTaylor K, Britton P, O\u0026rsquo;Keeffe S, Wallis MG. Quantification of the UK 5-point breast imaging classification and mapping to BI-RADS to facilitate comparison with international literature. British Journal of Radiology [Internet]. 2011;84(1007):1005\u0026ndash;10. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1259/bjr/48490964\u003c/span\u003e\u003cspan address=\"10.1259/bjr/48490964\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDonner A, Rotondi MA. Sample size requirements for interval estimation of the kappa statistic for interobserver agreement studies with a binary outcome and multiple raters. International Journal of Biostatistics [Internet]. 2010 [cited 2023 Jun 18];6(1). Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.2202/1557-4679.1275\u003c/span\u003e\u003cspan address=\"10.2202/1557-4679.1275\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSuzuki S. Conditional Relative Odds Ratio and Comparison of Accuracy of Diagnostic Tests Based on 2\u0026times;2 Tables. J Epidemiol [Internet]. 2006 [cited 2023 Oct 1];16(4):145\u0026ndash;53. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.2188/jea.16.145\u003c/span\u003e\u003cspan address=\"10.2188/jea.16.145\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSickles E, D\u0026rsquo;Orsi C. ACR BI-RADS\u0026reg; Follow-up and Outcome Monitoring. In: ACR BI-RADS\u0026reg; Atlas, Breast Imaging Reporting and Data System [Internet]. 5th ed. Reston, VA: American College of Radiology; 2013. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.acr.org/Clinical-Resources/Reporting-and-Data-Systems/Bi-Rads/Permissions\u003c/span\u003e\u003cspan address=\"https://www.acr.org/Clinical-Resources/Reporting-and-Data-Systems/Bi-Rads/Permissions\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTaylor-Phillips S, Wallis MG, Jenkinson D, Adekanmbi V, Parsons H, Dunn J, et al. Effect of using the same vs different order for second readings of screening mammograms on rates of breast cancer detection: A randomized clinical trial. JAMA - Journal of the American Medical Association [Internet]. 2016;315(18):1956\u0026ndash;65. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1001/jama.2016.5257\u003c/span\u003e\u003cspan address=\"10.1001/jama.2016.5257\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBackmann HA, Larsen M, Danielsen AS, Hofvind S. Does it matter for the radiologists\u0026rsquo; performance whether they read short or long batches in organized mammographic screening? Eur Radiol [Internet]. 2021;31:9548\u0026ndash;55. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s00330-021-08010-9\u003c/span\u003e\u003cspan address=\"10.1007/s00330-021-08010-9\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen Y, Sudin ES, Partridge GJ, Taib AG, Darker IT, Phillips P, et al. Measuring reader fatigue in the interpretation of screening digital breast tomosynthesis (DBT). British Journal of Radiology [Internet]. 2023 Feb 1 [cited 2023 May 17];96(1143). Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1259/bjr.20220629\u003c/span\u003e\u003cspan address=\"10.1259/bjr.20220629\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePartridge G, Darker I, James J, Michell M, Chen Y. How long does it take to read a mammogram. In: Symposium mammographicum abstract booklet [Internet]. 2023 [cited 2023 Oct 2]. p. 13\u0026ndash;13. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://sympmamm.org.uk/wp-content/uploads/2023/07/Symposium-Mammographicum-Abstracts-2023.pdf\u003c/span\u003e\u003cspan address=\"https://sympmamm.org.uk/wp-content/uploads/2023/07/Symposium-Mammographicum-Abstracts-2023.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTaylor-Phillips S, Wallis M, Gale AG. Should previous mammograms be digitised in the transition to digital mammography ? Breast [Internet]. 2009 [cited 2023 Oct 14];19:1890\u0026ndash;6. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s00330-009-1366-x\u003c/span\u003e\u003cspan address=\"10.1007/s00330-009-1366-x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDahlblom V, Dustler M, Tingberg A, Zackrisson S. Breast cancer screening with digital breast tomosynthesis: comparison of different reading strategies implementing artificial intelligence. Eur Radiol [Internet]. 2023;33:3754\u0026ndash;65. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s00330-022-09316-y\u003c/span\u003e\u003cspan address=\"10.1007/s00330-022-09316-y\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRaya-Povedano JL, Romero-Mart\u0026iacute;n S, El\u0026iacute;as-Cabot E, Gubern-M\u0026eacute;rida A, Rodr\u0026iacute;guez-Ruiz A, \u0026Aacute;lvarez-Benito M. AI-based Strategies to Reduce Workload in Breast Cancer Screening with Mammography and Tomosynthesis: A Retrospective Evaluation. Radiology [Internet]. 2021 Jul 1 [cited 2023 Jun 13];300(1):57\u0026ndash;65. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1148/radiol.2021203555\u003c/span\u003e\u003cspan address=\"10.1148/radiol.2021203555\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGur D, Cohen CS, Hakim CM, Hardesty LA, Ganott MA, Perrin RL, et al. The \u0026ldquo;Laboratory Effect\u0026rdquo;: Comparing Radiologists\u0026rsquo; Performance and Variability during Prospective Clinical and Laboratory Mammography. Radiology [Internet]. 2008;249(1):47\u0026ndash;53. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1148/radiol.2491072025\u003c/span\u003e\u003cspan address=\"10.1148/radiol.2491072025\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLee JM, Ichikawa L, Valencia E, Miglioretti DL, Wernli K, Kerlikowske K, et al. Performance Benchmarks for Screening Breast MR Imaging in Community Practice. Radiology [Internet]. 2017;285(1):44\u0026ndash;52. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1148/radiol.2017162033\u003c/span\u003e\u003cspan address=\"10.1148/radiol.2017162033\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSaadatmand S, Obdeijn IM, Rutgers EJ, Oosterwijk JC, Tollenaar RA, Woldringh GH, et al. Survival benefit in women with BRCA1 mutation or familial risk in the MRI screening study (MRISC). Int J Cancer [Internet]. 2015;137(7):1729\u0026ndash;38. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/ijc.29534\u003c/span\u003e\u003cspan address=\"10.1002/ijc.29534\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEvans DG, Harkness EF, Howell A, Wilson M, Hurley E, Holmen MM, et al. Intensive breast screening in BRCA2 mutation carriers is associated with reduced breast cancer specific and all cause mortality. Hered Cancer Clin Pract [Internet]. 2016;14(8):1\u0026ndash;8. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://dx.doi.org/10.1186/s13053-016-0048-3\u003c/span\u003e\u003cspan address=\"10.1186/s13053-016-0048-3\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen Y, James JJ, Michalopoulou E, Darker IT, Jenkins J. Performance of Radiologists and Radiographers in Double Reading Mammograms: The UK National Health Service Breast Screening Program. Radiology [Internet]. 2023 Jan 1 [cited 2023 Jun 12];306(1):102\u0026ndash;9. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1148/radiol.212951\u003c/span\u003e\u003cspan address=\"10.1148/radiol.212951\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNational Breast Imaging Academy. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://nationalbreastimagingacademy.org/breast-clinicians/about-breast-clinicians/breast-clinician-credential/\u003c/span\u003e\u003cspan address=\"https://nationalbreastimagingacademy.org/breast-clinicians/about-breast-clinicians/breast-clinician-credential/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. 2019. Breast Clinician Credential.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNHS Health Education England. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://advanced-practice.hee.nhs.uk\u003c/span\u003e\u003cspan address=\"https://advanced-practice.hee.nhs.uk\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. 2023. Advanced Practitioner Training.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"breast-cancer-research","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"brcr","sideBox":"Learn more about [Breast Cancer Research](http://breast-cancer-research.biomedcentral.com)","snPcode":"13058","submissionUrl":"https://submission.nature.com/new-submission/13058/3","title":"Breast Cancer Research","twitterHandle":"@BCRJournal","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"FAST MRI, abbreviated breast MRI, breast cancer, screening, formative assessment, medical education, diagnostic accuracy, e-learning","lastPublishedDoi":"10.21203/rs.3.rs-3881738/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-3881738/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground:\u003c/strong\u003e Abbreviated breast MRI (FAST MRI) is being introduced into clinical practice. Specificity optimisation is essential to minimise harm through false positive results for populations with low pre-test probability. This study aimed to optimise diagnostic accuracy through the adaptation of a FAST MRI interpretation-training programme.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods:\u003c/strong\u003e A FAST MRI interpretation-training programme was adapted to provide additional training during the assessment task (interpretation of an enriched dataset of 125 FAST MRI scans) by giving readers feedback about the true outcome of each scan immediately after each scan was interpreted (formative assessment). \u0026nbsp;The training programme was additionally adapted for remote e-learning delivery.\u003c/p\u003e\n\u003cp\u003eStudy design: prospective, blinded interpretation of an enriched dataset by multiple readers.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e 43 mammogram readers completed the training, 22 who interpreted breast MRI in their clinical role (Group 1) and 21 who did not (Group 2).\u003c/p\u003e\n\u003cp\u003eOverall sensitivity was 83% (95%CI 81-84%; 1994/2408), specificity 94% (95%CI 93–94%; 7806/8338), readers’ agreement with the true outcome kappa = 0.75 (95%CI 0.74-0.77) and diagnostic odds ratio = 70.67 (95%CI 61.59-81.09). Group 1 readers showed similar sensitivity (84%) to Group 2 (82% p=0.14), but slightly higher specificity (94% v. 93%, p=0.001).\u003c/p\u003e\n\u003cp\u003eConcordance with the ground truth increased significantly with the number of FAST MRI scans read through the formative assessment task (p=0.002) but by differing amounts depending on whether or not a reader had previously attended FAST MRI training (interaction p=0.02). Concordance with the ground truth was significantly associated with reading batch size (p=0.02), tending to worsen when more than 50 scans were read per batch.\u003c/p\u003e\n\u003cp\u003eGroup 1 took a median of 56 seconds (range 8-47466) to interpret each FAST MRI scan compared with 78 seconds (14-22830, p \u0026lt;0.0001) for Group 2.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions:\u003c/strong\u003e Provision of immediate feedback to mammogram readers during the assessment test set reading task increased specificity for FAST MRI interpretation and achieved high diagnostic accuracy.\u003c/p\u003e\n\u003cp\u003eOptimal reading-batch size for FAST MRI was 50 reads per batch.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTrial registration\u003c/strong\u003e (25/09/2019)\u003cstrong\u003e:\u003c/strong\u003e ISRCTN16624917\u003c/p\u003e","manuscriptTitle":"Optimising the diagnostic accuracy of First post-contrAst SubtracTed breast MRI (FAST MRI) through interpretation-training: a multicentre e-learning study, mapping the learning curve of NHS Breast Screening Programme (NHSBSP) mammogram readers using an enriched dataset","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-01-25 19:59:49","doi":"10.21203/rs.3.rs-3881738/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-02-25T19:08:13+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-02-19T14:31:44+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"cc7715e5-5ac2-4c09-8f0c-b24ba8aa2116","date":"2024-02-05T14:32:15+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-02-05T14:17:00+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-01-22T05:56:05+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-01-22T03:14:43+00:00","index":"","fulltext":""},{"type":"submitted","content":"Breast Cancer Research","date":"2024-01-20T14:06:48+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"breast-cancer-research","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"brcr","sideBox":"Learn more about [Breast Cancer Research](http://breast-cancer-research.biomedcentral.com)","snPcode":"13058","submissionUrl":"https://submission.nature.com/new-submission/13058/3","title":"Breast Cancer Research","twitterHandle":"@BCRJournal","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2eaf306a-237f-458c-8b8b-66052940cf4d","owner":[],"postedDate":"January 25th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2024-05-18T15:25:13+00:00","versionOfRecord":[],"versionCreatedAt":"2024-01-25 19:59:49","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-3881738","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-3881738","identity":"rs-3881738","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}