Incorporating Desirable Difficulties into the design of digital learning: A think-aloud study

Incorporating Desirable Difficulties into the design of digital learning: A think-aloud study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Incorporating Desirable Difficulties into the design of digital learning: A think-aloud study Kathlyn Tsou, Siew Ping Han This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7421926/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 8 You are reading this latest preprint version Abstract Background The use of digital learning platforms is becoming increasingly common in health professions education, but there are few studies evaluating their effectiveness based on cognitive science principles. Here, we draw upon well-established learning concepts grounded in cognitive science – desirable difficulties, productive struggle and cognitive load theory – to explore whether and how digital learning platforms can enhance learning. Our study aims were to i) identify points of struggle when participants processed learning material on a digital platform compared to traditional PDF learning materials and ii) explore how design factors influenced their subsequent responses. Methods Using in-depth think-aloud interviews, we compared how medical student participants engaged with learning material in traditional PDF format versus similar content on an online learning platform (five participants, two interviews each for a total of ten interviews). Participants were instructed to navigate the learning material as they would in their usual practice while verbalizing their thoughts during the process. Interviews were conducted on Zoom and video-recorded. Transcripts and videos were analysed using thematic analysis. Results We identified three themes centred around non-struggle, generative struggle and non-generative struggle. In the absence of struggle, learners tended to learn superficially and remained disengaged from the material. Desirable difficulties, such as in the form of online quizzes, enhanced learning through introducing points of struggle that led to deeper processing of information, which we term generative struggle. However, struggle that resulted from increased extraneous load due to flawed design was counter-productive and led to disengagement, which we term non-generative struggle. Conclusions Based on our work, we propose a model for how increased cognitive load due to desirable difficulties promotes generative processing and greater engagement with the learning material. This work can guide the future design and evaluation of digital learning platforms for more effective learning based on cognitive science principles. Digital learning educational technology desirable difficulties productive struggle cognitive load theory Background The use of digital learning platforms in health professions education is becoming increasingly widespread, but evidence of their effectiveness is still limited ( 1 ). While they can offer scalability and accessibility, cost estimates range widely and often do not capture hidden implementation and production costs ( 2 ). Further, there is limited high-quality evidence of their educational impact and return on investment compared to traditional teaching methods ( 3 ). Therefore, there is a growing call for more evidence-based evaluation of whether and how the use of learning technology in health professions education enhances achievement of learning goals ( 4 , 5 ). An area of research into effective strategies for education supported by rigorous empirical evidence is the idea of Desirable Difficulty (DD). DD strategies are based on cognitive science principles that effective learning requires memory to be both available (encoded into long-term memory) and accessible (can be retrieved from long-term memory). DD postulates that more difficult learning processes can enhance learning outcomes and increase the durability and transferability of knowledge. The benefits of DD learning strategies have been demonstrated across a range of learners including medical, pharmacy, surgery and anatomy students (reviewed in ( 6 )). However, because learning is effortful and delayed, learners often do not appreciate this approach, even when presented with evidence of its effectiveness ( 7 ). A related concept well-established in mathematics education is productive struggle, which suggests that the increased cognitive effort required to solve complex problems leads to deeper understanding ( 8 ). Importantly, for the struggle to be productive, the task should be designed so that the learner is challenged but not overwhelmed by the difficulty of the task ( 9 ). In other words, the level of difficulty must not exceed the learners’ maximum cognitive capacity, otherwise their learning would decrease due to frustration and reduced motivation to continue putting in effort to learn ( 10 , 11 ). Cognitive load theory states that cognitive capacity is exceeded when there is too much extraneous (distractions) or intrinsic (complex information) load, such that learners lack cognitive capacity for germane load (actual work of learning) ( 12 ). Germane cognitive load is analogous to generative processing, which is the cognitive processing that makes sense of content. Optimal generative processing occurs when learners exert and maintain effort to make sense of content at a sufficient level of intensity ( 13 ). More specifically, the Cognitive Theory of Multimedia Learning suggests that learning can be enhanced through deliberate instructional design decisions to reduce extraneous load and free up cognitive capacity for generative processing ( 14 , 15 ). Although the importance of introducing optimal cognitive load in health professions education has been established for over a decade ( 10 , 14 – 16 ), its application in the design of learning materials for medical students remains limited. Most studies evaluating the educational impact of learning technology rely on tests of short-term retention, which, while no doubt important and informative, do not provide insight into underlying cognitive processes. One way to make some aspects of cognitive processing during a task observable is through the think-aloud approach. As participants engage in a task, they are instructed to verbalise whatever they are thinking with minimal direction from the interviewer. The think-aloud approach is particularly useful in identifying areas where participants struggle with the task and the factors influencing their response ( 17 , 18 ) and can be applied to the investigation of how participants process information on digital interfaces ( 19 , 20 ). Therefore, the think-aloud approach would be suitable for investigating if and how learning technology stimulates cognitive processes such as productive struggle and generative processing. In this study, we conducted think-aloud interviews with medical undergraduates to compare their cognitive processes while reading a science practical manual in PDF format and from a digital learning platform. Our research questions were: What are the points of struggle when participants process information on a digital learning platform as compared to a PDF? What design factors influence how participants respond to the points of struggle? Through this work, we aim to identify factors that enhance the learning value of digital learning platforms and develop a cognitive model to guide the future design of more effective digital learning and other learning technologies. Methods Context The study was on two versions of a manual (PDF vs digital platform) used for a science practical on spirometry as part of the undergraduate MBBS year 1 curriculum at the Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore. Students are expected to read the manual, which includes background theoretical information, instructions on how to carry out the practical and data collection forms, prior to the practical. The science practical team developed a new digital version of the manual on the Lt online learning platform (AD Instruments) with similar content but additional features such as an interactive quiz that tested some of the concepts covered in the rest of the content. Participants Year 2 students were recruited via convenience sampling. Participants were offered two SGD 10 vouchers (for two interview sessions) as reimbursement. Informed consent was obtained from all participants prior to the interviews. Data collection During think-aloud interviews, participants were asked to verbalize their thoughts as they navigated the two versions of the science practical manual. Participants were interviewed twice, once while navigating the PDF version and then again approximately 3–4 weeks later while navigating the platform version. Interviews were conducted according to an interview guide and a think-aloud interview protocol (Additional file 1) which closely matched the speech communication theory-based protocol described by Boren and Ramey ( 21 ). This protocol was selected from a pilot trial of interview protocols on a volunteer. All interviews were carried out using video conferencing platform Zoom. Participants were asked to share their Zoom screens so that their mouse movements and what they were viewing in the PDF or platform interface could be captured. Interviews were recorded and audio transcripts were anonymized prior to analysis. Data analysis We took a deductive-inductive approach and analysed the transcripts and videos using thematic analysis ( 22 ). Videos were included in the analysis process as they revealed key non-textual data such as where participants clicked or when they paused to search for information ( 23 ). Briefly, we familiarized ourselves with the transcripts and videos through repeated reading and viewing. We developed an initial codebook based on the Cognitive Theory of Multimedia Learning ( 13 ). Initial analysis suggested that generative processing and its link to productive struggle was the most relevant for the analysis. Therefore, we developed a final codebook based on productive struggle (Additional file 2, ( 8 )). Microsoft Word and Excel were used for data management and coding. Discrepancies between coders were resolved through discussion and mutual consensus. Reflexivity Throughout this qualitative research process, we were cognisant of how our backgrounds might affect our relationship with the data. KT was, at the time of data collection, a year 4 medical student who approached the project more from a learner’s perspective. HSP is a medical educator and researcher with a background in molecular biology, whose views may have been more aligned with that of faculty. Together, we provided both “insider” and “outsider” views of how teaching materials are experienced by students. Results A total of ten interviews were conducted with five Year 2 students. Interview lengths ranged from 24–49 min (median 38 min, total 380 min). Participants have been anonymised (P1-5) followed by whether the interview was based on the PDF or platform version of the manual. We report verbatim quotes with ellipses (…) to indicate where less relevant text has been cut out and square brackets to indicate any non-verbatim explanatory text. We identified three themes centred around non-struggle, generative struggle and non-generative struggle. Non-struggle leads to low engagement For both the PDF and platform-based format, we observed that participants tended to skim through most of the learning material without properly engaging with it. “I usually just skim because I, I don't understand the first part, usually like the background and all the equations, so I just skip to the what we're supposed to do the next day part” (P3, PDF) While participants felt the platform-based manual was “more reader-friendly”, “clearer” and “very structured”, the reduced effort of reading did not seem to improve their engagement with the content, with most participants scrolling quickly across the pages with few pauses or verbalization of thoughts. In fact, the use of formatting to signpost important points, while enhancing clarity, sometimes led to participants disregarding content that was not formatted in that way. “OK, I guess the other two aren’t that important because there's no bolded words” (P3, platform) Generative struggle enhances engagement In our first round of coding focusing on cognitive load, we found that participants generally preferred the platform-based manual over the PDF manual because it reduced perceived cognitive effort. However, we realized that moments of deeper engagement with the learning material occurred at points that suggested the participants were experiencing higher cognitive load, for example when they paused to think, expressed uncertainty by questioning themselves or navigated to previously read material to check their understanding. This was most evident when the participants attempted the quizzes in the platform-based manual. There was increased verbalization of their thought processes, wherein they expressed doubt in their current understanding and re-read content more slowly to fill in the gaps in their knowledge. “I think, then restricted should be lower because for obstruction, I think it’s, wait, wait no, wrong. I think should be this because restriction is when you inhale and exhale with difficulty while obstruction is only when you exhale there’s difficulty, so I think restriction should have lower vital capacity, then yeah, I think this should be normal, obstructed and treated” (P2, platform) “for instance, right, just now I did capacity wrong, so I can go back and refer to what inspiration capacity is” (P4, platform) Participants appreciated the fact that attempting the quiz made them think and check their understanding of the content. “I feel like it made me, like, have to think on my own, you know how people talk about, like, passive recall active recall, like this actually helped me to make sure that I am like my mind is engaged when I'm doing this and I know what they're talking about” (P5, platform) While infrequent, there were also instances where participants reading the PDF manual appeared to struggle with the content, which led them to process it more deeply. “The resulting drop in pressure is measured by using differential pressure trends. Oh, don't really know what that means… [mumbles while reading the content]. Oh, OK, can be used to confirm a diagnosis and a convenient means of monitoring progress” (P5, PDF) Overall, we found that participants tended to exhibit signs of engagement with the content when increased cognitive load caused them to pause to think, be it due to a quiz or their own realization of knowledge gaps, rather than skimming through the manual quickly. Therefore, we considered this to be a kind of struggle that led to generative processing. Non-generative struggle leads to disengagement However, increased cognitive effort could also cause participants to avoid reading parts of the learning material. This effort could be due to the perceived complexity of the content or challenges in navigating through the manual. “Maybe I skip through something like P O2 because I don’t really understand what’s P O2 , maybe I read too fast just now” (P1, PDF) Further, participants sometimes missed key information due to challenges faced in navigating the manual. [Participant missed reading large section of text due to clicking on hyperlink earlier] Interviewer: "Are you able to go back to go, will you be able to go back to those steps?" P5: "Oh my gosh, yeah, I just realised I missed this. This one right? OK. Teaching methods. Yes.” (P5, PDF) While participants generally found the platform-based manual easier to navigate, there were still issues with transiting between pages and locating the correct icons to click. Thus, while excessive complexity of content (intrinsic load) and instructional design flaws (extrinsic load) appeared to cause increased cognitive load that made participants pause to think, this did not enhance understanding of content and in some cases, led to disengagement from the content. Therefore, we considered this to be a kind of struggle that was non-generative. Discussion Summary of findings Overall, our findings revealed two types of points of struggle. Firstly, design flaws and excessively complex content caused excess cognitive load that was counter-productive and caused participants to skip or miss information. Secondly, quizzes or engagement in understanding a challenging piece of information increased cognitive effort but also led to deeper engagement with the material. The affordances of the platform-based manual reduced cognitive load by making navigation of content easier and chunking complex content. However, some cognitive load was still required to drive generative processing and higher order learning. Comparison with the literature Cognitive load theory postulates that learning is less effective if cognitive load exceeds the learner’s working memory (reviewed in ( 12 , 14 , 15 )). The three types of cognitive load are extraneous load, intrinsic load and germane load. Extraneous load utilizes working memory but does not contribute to learning. For instance, visual overload may result from excessive text that is presented in a disorganized manner and requires the learner to search for information and piece it together. In our study, this was exemplified in the PDF version of the manual when participants had to read through copious amounts of text and were confused by hyperlinks that brought them to another part of the PDF, such that they lost track of where they had read till. Intrinsic load is caused by the inherent complexity of the task. It can be managed by, for example, chunking complex content into smaller pieces, as was the case for the platform version of the manual. The design of digital education tools, especially when based on student preferences for lower effort when learning, tends to focus on the reduction of extraneous and intrinsic load through better interface design and structuring of content. However, effective learning in which information is assimilated into long term memory is dependent on germane load, which requires deliberate effort and concentration from the learner ( 15 ). In other words, learners must be given appropriately challenging tasks, i.e. DDs, that activate their prior knowledge and engage them in exploration to complete the task, so as to develop conceptual knowledge and transfer through productive struggle ( 8 ). In our study, this tended to occur when participants were presented with quizzes that engaged their attention and activated retrieval practice. Through verbalizing their thoughts and revisiting the learning material, they were able to process the content in a deeper and more integrated manner. Proposed cognitive model of desirable difficulties in digital learning Based on the above, we propose the following model of how DD shapes learner engagement in digital learning: In the absence of DD and low cognitive load, learners tend to learn superficially and remain disengaged from the material. High cognitive load primarily due to extraneous load, rather than germane load introduced by DD, increases cognitive effort without contributing to learning. This results in non-generative struggle, which may in turn lead to frustration and disengagement as well. High (but not excessive) cognitive load resulting from DDs causes generative struggle in learners. Increased cognitive effort is directed to higher-order thinking and more effective learning. The above points are summarized in Table 1 . Table 1 Relationship between cognitive load and generative struggle Lack of DD Presence of DD Low cognitive load Disengagement High cognitive load Non-generative struggle Generative struggle Design factors that may contribute to high cognitive load through extraneous load (which should be minimized) include excessive text, spatially disorganized content, unclear formatting and redundant information. Design factors that increase germane load (which should be enhanced) include incorporation of quizzes and other interactive elements that activate retrieval practice. Ideally, quizzes should not just give direct answers but instead provide elaborated feedback to prompt further generative processing in learners ( 24 ). In our study, we have focused on design factors. Learner factors such as prior level of knowledge, motivation and learning approaches also affect engagement with DD ( 25 ) and warrant further investigation on how they fit into this model. Implications for practice and research With the increasing adoption of digital tools in health professions education, there is great potential for utilizing digital affordances to guide learners in analysing, applying and transferring knowledge to achieve higher-order learning outcomes. DD strategies such as retrieval practice, spaced practice and interleaved practice can easily be applied through quizzes and practice tests on digital platforms ( 6 ). The timing and type of feedback on quiz or test performance can be optimized through adaptive systems to enhance long-term retention and deeper learning ( 24 ). Our work adds to the literature by highlighting the importance of incorporating design elements that introduce DDs, create productive struggle and promote generative processing for deeper learning. It is important to raise awareness of DD and productive struggle principles not only amongst medical educators, but medical students as well. Students often fall prey to the “fluency illusion”, wherein they prefer passive learning strategies such as rereading or highlighting because such strategies feel easier and aid short-term memory, which creates overconfidence in their ability to retain knowledge ( 7 ). Conversely, students tend to reject strategies such as retrieval practice and delayed feedback, despite the robust empirical evidence of their effectiveness, because they require more effort and generate struggle ( 26 , 27 ). Thus, the conventional practice of evaluating digital (and other) educational tools based on institution-wide surveys of student preferences may not provide accurate information on their actual effectiveness. Instead, evaluation of digital education tools needs to move away from student ratings or tests of short-term retention and instead explore whether and how these tools promote higher-order thinking. This may require more individualised, in-depth studies using methods such as think-aloud interviews to reveal cognitive processes occurring in learners. Further, motivational strategies such as providing an appropriate level of challenge, learning choices and timely feedback can increase students’ acceptance of DD and other effective learning strategies ( 28 , 29 ). Here, the potential for digital tools to be personalised and flexible, especially coupled with the new affordances introduced by genAI for adaptive learning ( 30 , 31 ), can be useful in developing customised learning resources that match the abilities and preferences of students and therefore enhance deeper learning. Limitations As an exploratory study, our sample size was relatively small. Still, we have confidence in our conclusions as the research question was focused, the target group was specific, the study was grounded in theory and the interview data was rich ( 32 ). Further, a previous study has found that 80% of findings can be detected with just four or five think-aloud participants ( 33 ). The process of verbalising thoughts in a think-aloud protocol increases cognitive load for participants, which may have affected how they interacted with the material. Nevertheless, the material used in the think-aloud session was designed for year 1 students and therefore should not have placed excessive cognitive strain on the year 2 participants. Conclusion We propose a model for how the introduction of DDs into the design of digital learning platforms can enhance productive struggle, leading to generative processing that enhances learning. In doing so, we identify design factors that may affect disengagement or engagement with the learning material. We also demonstrate how think-aloud interviews can provide insight into the cognitive processes of digital learning platform users to investigate if and how higher-order thinking occurs. Future design of digital learning tools should aim to incorporate DDs with an appropriate level of challenge to enhance generative struggle and therefore make learning more effective. Abbreviations DD – desirable difficulties Declarations Ethics approval and consent to participate Ethical approval was obtained from the Nanyang Technological University Institutional Review Board (IRB-2021-1070). Freely-given informed consent was obtained from all participants. Consent for publication Not applicable. Availability of data and materials The datasets used during the current study are available from the corresponding author on reasonable request. Competing interests The authors declare that they have no competing interests. Authors’ contributions KT designed the study, conducted the interviews and performed the analysis. SPH designed the study, performed the analysis and wrote the manuscript. All authors read and approved of the final manuscript. Acknowledgements We would like to thank Darren Lim for technical support and access to learning materials used in the data collection. Funding declaration Funding was provided by the Nanyang Technological University Edex grant. References Pei L, Wu H. Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis. Medical Education Online. 2019 Jan 1;24(1):1666538. Rees CE, Nguyen VNB, Foo J, Edouard V, Maloney S, Palermo C. Balancing the effectiveness and cost of online education: A preliminary realist economic evaluation. Medical Teacher. 2022 Sep 2;44(9):977–85. Meinert E, Eerens J, Banks C, Maloney S, Rivers G, Ilic D, et al. Exploring the Cost of eLearning in Health Professions Education: Scoping Review. JMIR Med Educ. 2021 Mar 11;7(1):e13681. Han SP, Ng O. Teaching touch with technology: Realism and pedagogical intent in digital simulation. Medical Education. 2025 Aug;59(8):792-4. McGaghie WC, Barsuk JH, Wayne DB, Issenberg SB. Powerful medical education improves health care quality and return on investment. Medical Teacher. 2024 Jan 2;46(1):46–58. Nelson A, Eliasz KL. Desirable Difficulty: Theory and application of intentionally challenging learning. Medical Education. 2023 Feb;57(2):123–30. Biwer F, Egbrink MGAO, Aalten P, De Bruin ABH. Fostering effective learning strategies in higher education—A mixed-methods study. Journal of Applied Research in Memory and Cognition. 2020 Jun;9(2):186–203. Kapur M. Examining Productive Failure, Productive Success, Unproductive Failure, and Unproductive Success in Learning. Educational Psychologist. 2016 Apr 2;51(2):289–99. Young JR, Bevan D, Sanders M. How Productive is the Productive Struggle? Lessons Learned from a Scoping Review. IJEMST. 2023 Nov 23;12(2):470–95. Guadagnoli M, Morin M, Dubrowski A. The application of the challenge point framework in medical education. Medical Education. 2012 May;46(5):447–53. Guadagnoli MA, Lee TD. Challenge Point: A Framework for Conceptualizing the Effects of Various Practice Conditions in Motor Learning. Journal of Motor Behavior. 2004 Jul;36(2):212–24. Paas F, Sweller J. Implications of Cognitive Load Theory for Multimedia Learning. In: Mayer RE, editor. The Cambridge Handbook of Multimedia Learning. 2nd ed. Cambridge University Press; 2014. p. 27–42. Mayer RE. Cognitive Theory of Multimedia Learning. In: Mayer RE, editor. The Cambridge Handbook of Multimedia Learning. 2nd ed. Cambridge University Press; 2014. p. 43–71. Van Merrienboer J, Sweller J. Cognitive load theory in health professional education: design principles and strategies. Medical Education. 2010 Jan;44(1):85–93. Young JQ, Van Merrienboer J, Durning S, Ten Cate O. Cognitive Load Theory: Implications for medical education: AMEE Guide No. 86. Medical Teacher. 2014 May;36(5):371–84. Dobson JL, Linderholm T. The effect of selected “desirable difficulties” on the ability to recall anatomy information. Anatomical Sciences Ed. 2015 Sep;8(5):395–403. Olmsted-Hawala EL, Murphy ED, Hawala S, Ashenfelter KT. Think-aloud protocols: a comparison of three think-aloud protocols for use in testing data-dissemination web sites for usability. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Atlanta Georgia USA: ACM; 2010. p. 2381–90. Wolcott MD, Lobczowski NG. Using cognitive interviews and think-aloud protocols to understand thought processes. Currents in Pharmacy Teaching and Learning. 2021 Feb;13(2):181–8. Eveland WP, Dunwoody S. Examining Information Processing on the World Wide Web Using Think Aloud Protocols. Media Psychology. 2000 May;2(3):219–44. Jaspers M, Steen T, Bos C, Geenen M. The think aloud method: a guide to user interface design. International Journal of Medical Informatics. 2004 Nov;73(11–12):781–95. Boren T, Ramey J. Thinking aloud: reconciling theory and practice. IEEE Trans Profess Commun. 2000;43(3):261–78. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006 Jan;3(2):77–101. Noushad B, Van Gerven PWM, De Bruin ABH. Twelve tips for applying the think-aloud method to capture cognitive processes. Medical Teacher. 2024 Jul 2;46(7):892–7. Attali Y, Van Der Kleij F. Effects of feedback elaboration and feedback timing during computer-based practice in mathematics problem solving. Computers & Education. 2017 Jul;110:154–69. De Bruin ABH, Biwer F, Hui L, Onan E, David L, Wiradhany W. Worth the Effort: the Start and Stick to Desirable Difficulties (S2D2) Framework. Educ Psychol Rev. 2023 Jun;35(2):41. https://doi.org/10.1007/s10648-023-09766-w Hui L, De Bruin ABH, Donkers J, Van Merriënboer JJG. Why students do (or do not) choose retrieval practice: Their perceptions of mental effort during task performance matter. Applied Cognitive Psychology. 2022 Mar;36(2):433–44. Mullet HG, Butler AC, Verdin B, Von Borries R, Marsh EJ. Delaying feedback promotes transfer of knowledge despite student preferences to receive feedback immediately. Journal of Applied Research in Memory and Cognition. 2014 Sep;3(3):222–9. Hui L, De Bruin ABH, Donkers J, Van Merriënboer JJG. Does Individual Performance Feedback Increase the Use of Retrieval Practice? Educ Psychol Rev. 2021 Dec;33(4):1835–57. Zepeda CD, Martin RS, Butler AC. Motivational strategies to engage learners in desirable difficulties. Journal of Applied Research in Memory and Cognition. 2020 Dec;9(4):468–74. Masters K, Herrmann-Werner A, Festl-Wietek T, Taylor D. Preparing for Artificial General Intelligence (AGI) in Health Professions Education: AMEE Guide No. 172. Medical Teacher. 2024 Oct 2;46(10):1258–71. Røe Y, Wojniusz S, Bjerke AH. The Digital Transformation of Higher Education Teaching: Four Pedagogical Prescriptions to Move Active Learning Pedagogy Forward. Front Educ. 2022 Jan 14;6:784701. Malterud K, Siersma VD, Guassora AD. Sample Size in Qualitative Interview Studies: Guided by Information Power. Qual Health Res. 2016 Nov;26(13):1753–60. Virzi RA. Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? Hum Factors. 1992 Aug;34(4):457–68. Additional Declarations No competing interests reported. Supplementary Files Additionalfile1.docx Additionalfile2.docx Cite Share Download PDF Status: Under Review Version 1 posted Reviews received at journal 14 Oct, 2025 Reviewers agreed at journal 07 Oct, 2025 Reviewers agreed at journal 06 Oct, 2025 Reviewers invited by journal 06 Oct, 2025 Editor invited by journal 10 Sep, 2025 Editor assigned by journal 09 Sep, 2025 Submission checks completed at journal 09 Sep, 2025 First submitted to journal 20 Aug, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7421926","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":529961496,"identity":"939d28de-97be-4d55-a387-900def3a7a97","order_by":0,"name":"Kathlyn Tsou","email":"","orcid":"","institution":"Lee Kong Chian School of Medicine, Nanyang Technological University","correspondingAuthor":false,"prefix":"","firstName":"Kathlyn","middleName":"","lastName":"Tsou","suffix":""},{"id":529961497,"identity":"940f0e4d-94f2-4157-8993-0c4e80bfce05","order_by":1,"name":"Siew Ping Han","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA/klEQVRIiWNgGAWjYDACZhBhAMQSQPwBImZAvBbGGRDVBLTAAVALMw8xWuTbmY9JFxTcsWuQbn722HbHn8QG9uZtEgx/anFqMTjMliY9w+BZcoPMMXPj3DMGiQ08x8okGNuO49bCzGN2m8fgcDKDRIKZdG4bUItEjpkEY8Mx3A5r5v8G1ZL+TdoSpEX+jRnQYbi1MBzmYQNpsWMAGi7NCLaFB6iFrQafX8x/zzA4nMAmkVMm2dtmbNzGk1Zskdh2ALfD+g8/Ni74c9ieXyJ9m8TPNjnZfvbDG298+FOH22EMkNhMbIPx2EBEAsNhglrs0QXx2zIKRsEoGAUjCgAAVcBL9XqJ0ogAAAAASUVORK5CYII=","orcid":"","institution":"Lee Kong Chian School of Medicine, Nanyang Technological University","correspondingAuthor":true,"prefix":"","firstName":"Siew","middleName":"Ping","lastName":"Han","suffix":""}],"badges":[],"createdAt":"2025-08-21 03:53:12","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7421926/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7421926/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":93805651,"identity":"41cd7892-b82f-4fd0-a64a-631c00d655fc","added_by":"auto","created_at":"2025-10-17 18:02:27","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":42971,"visible":true,"origin":"","legend":"","description":"","filename":"20250903productivestruggleinonlineplatformunformatted.docx","url":"https://assets-eu.researchsquare.com/files/rs-7421926/v1/9b3f082716f8f168644cf057.docx"},{"id":93805654,"identity":"4da3ff9b-86c1-410d-82fd-8149376707fd","added_by":"auto","created_at":"2025-10-17 18:02:27","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5062,"visible":true,"origin":"","legend":"","description":"","filename":"fa382689a2f54fff8eb135ce821206cb.json","url":"https://assets-eu.researchsquare.com/files/rs-7421926/v1/025a2d3ddc7c8bc9a94f73f0.json"},{"id":93805655,"identity":"d0197650-ad34-4112-90c9-f3534a5fc76f","added_by":"auto","created_at":"2025-10-17 18:02:27","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":14208,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile1.docx","url":"https://assets-eu.researchsquare.com/files/rs-7421926/v1/2442f3497ab2ea7c7fe78187.docx"},{"id":93806354,"identity":"a44a7733-299f-4246-94b7-09280dad1c41","added_by":"auto","created_at":"2025-10-17 18:10:27","extension":"docx","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":16598,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile2.docx","url":"https://assets-eu.researchsquare.com/files/rs-7421926/v1/cfa876ffbebefb7ac5f49559.docx"},{"id":93805657,"identity":"4db77656-134d-406c-95f7-635dbb6281eb","added_by":"auto","created_at":"2025-10-17 18:02:27","extension":"xml","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":74543,"visible":true,"origin":"","legend":"","description":"","filename":"fa382689a2f54fff8eb135ce821206cb1enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-7421926/v1/0fce9da3e88af3511dae0820.xml"},{"id":93805659,"identity":"8625834c-4697-4649-8a28-c3eea4349e25","added_by":"auto","created_at":"2025-10-17 18:02:27","extension":"xml","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":72754,"visible":true,"origin":"","legend":"","description":"","filename":"fa382689a2f54fff8eb135ce821206cb1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7421926/v1/974fbd25649d194efdad573b.xml"},{"id":93805660,"identity":"3e419bf7-d94a-4335-9506-fe29b4993687","added_by":"auto","created_at":"2025-10-17 18:02:27","extension":"html","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":80373,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-7421926/v1/847d117a31558847542d35e0.html"},{"id":93806355,"identity":"b405a288-cb5c-429c-b5e2-c2707412e9bc","added_by":"auto","created_at":"2025-10-17 18:10:31","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":532760,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7421926/v1/440fd9e9-f494-413b-a1f4-beb69bc0a96f.pdf"},{"id":93805652,"identity":"de1fccf7-1e43-4649-bf2d-7e1d0ef88f86","added_by":"auto","created_at":"2025-10-17 18:02:27","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":14208,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile1.docx","url":"https://assets-eu.researchsquare.com/files/rs-7421926/v1/8fe78864f937df30659edb36.docx"},{"id":93806353,"identity":"8119b032-06a9-478c-b81b-aac74d0ae38c","added_by":"auto","created_at":"2025-10-17 18:10:27","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":16598,"visible":true,"origin":"","legend":"","description":"","filename":"Additionalfile2.docx","url":"https://assets-eu.researchsquare.com/files/rs-7421926/v1/5d408770c93facb0bd8166ce.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Incorporating Desirable Difficulties into the design of digital learning: A think-aloud study","fulltext":[{"header":"Background","content":"\u003cp\u003eThe use of digital learning platforms in health professions education is becoming increasingly widespread, but evidence of their effectiveness is still limited (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e). While they can offer scalability and accessibility, cost estimates range widely and often do not capture hidden implementation and production costs (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e). Further, there is limited high-quality evidence of their educational impact and return on investment compared to traditional teaching methods (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e). Therefore, there is a growing call for more evidence-based evaluation of whether and how the use of learning technology in health professions education enhances achievement of learning goals (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eAn area of research into effective strategies for education supported by rigorous empirical evidence is the idea of Desirable Difficulty (DD). DD strategies are based on cognitive science principles that effective learning requires memory to be both available (encoded into long-term memory) and accessible (can be retrieved from long-term memory). DD postulates that more difficult learning processes can enhance learning outcomes and increase the durability and transferability of knowledge. The benefits of DD learning strategies have been demonstrated across a range of learners including medical, pharmacy, surgery and anatomy students (reviewed in (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e)). However, because learning is effortful and delayed, learners often do not appreciate this approach, even when presented with evidence of its effectiveness (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eA related concept well-established in mathematics education is productive struggle, which suggests that the increased cognitive effort required to solve complex problems leads to deeper understanding (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e). Importantly, for the struggle to be productive, the task should be designed so that the learner is challenged but not overwhelmed by the difficulty of the task (\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e). In other words, the level of difficulty must not exceed the learners\u0026rsquo; maximum cognitive capacity, otherwise their learning would decrease due to frustration and reduced motivation to continue putting in effort to learn (\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eCognitive load theory states that cognitive capacity is exceeded when there is too much extraneous (distractions) or intrinsic (complex information) load, such that learners lack cognitive capacity for germane load (actual work of learning) (\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e). Germane cognitive load is analogous to generative processing, which is the cognitive processing that makes sense of content. Optimal generative processing occurs when learners exert and maintain effort to make sense of content at a sufficient level of intensity (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e). More specifically, the Cognitive Theory of Multimedia Learning suggests that learning can be enhanced through deliberate instructional design decisions to reduce extraneous load and free up cognitive capacity for generative processing (\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eAlthough the importance of introducing optimal cognitive load in health professions education has been established for over a decade (\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan additionalcitationids=\"CR15\" citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e), its application in the design of learning materials for medical students remains limited. Most studies evaluating the educational impact of learning technology rely on tests of short-term retention, which, while no doubt important and informative, do not provide insight into underlying cognitive processes. One way to make some aspects of cognitive processing during a task observable is through the think-aloud approach. As participants engage in a task, they are instructed to verbalise whatever they are thinking with minimal direction from the interviewer. The think-aloud approach is particularly useful in identifying areas where participants struggle with the task and the factors influencing their response (\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e, \u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e) and can be applied to the investigation of how participants process information on digital interfaces (\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e). Therefore, the think-aloud approach would be suitable for investigating if and how learning technology stimulates cognitive processes such as productive struggle and generative processing.\u003c/p\u003e\u003cp\u003eIn this study, we conducted think-aloud interviews with medical undergraduates to compare their cognitive processes while reading a science practical manual in PDF format and from a digital learning platform. Our research questions were:\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eWhat are the points of struggle when participants process information on a digital learning platform as compared to a PDF?\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eWhat design factors influence how participants respond to the points of struggle?\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cp\u003eThrough this work, we aim to identify factors that enhance the learning value of digital learning platforms and develop a cognitive model to guide the future design of more effective digital learning and other learning technologies.\u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\u003ch2\u003eContext\u003c/h2\u003e\u003cp\u003eThe study was on two versions of a manual (PDF vs digital platform) used for a science practical on spirometry as part of the undergraduate MBBS year 1 curriculum at the Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore. Students are expected to read the manual, which includes background theoretical information, instructions on how to carry out the practical and data collection forms, prior to the practical. The science practical team developed a new digital version of the manual on the Lt online learning platform (AD Instruments) with similar content but additional features such as an interactive quiz that tested some of the concepts covered in the rest of the content.\u003c/p\u003e\u003c/div\u003e\n\u003ch3\u003eParticipants\u003c/h3\u003e\n\u003cp\u003eYear 2 students were recruited via convenience sampling. Participants were offered two SGD 10 vouchers (for two interview sessions) as reimbursement. Informed consent was obtained from all participants prior to the interviews.\u003c/p\u003e\n\u003ch3\u003eData collection\u003c/h3\u003e\n\u003cp\u003eDuring think-aloud interviews, participants were asked to verbalize their thoughts as they navigated the two versions of the science practical manual. Participants were interviewed twice, once while navigating the PDF version and then again approximately 3\u0026ndash;4 weeks later while navigating the platform version. Interviews were conducted according to an interview guide and a think-aloud interview protocol (Additional file 1) which closely matched the speech communication theory-based protocol described by Boren and Ramey (\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e). This protocol was selected from a pilot trial of interview protocols on a volunteer. All interviews were carried out using video conferencing platform Zoom. Participants were asked to share their Zoom screens so that their mouse movements and what they were viewing in the PDF or platform interface could be captured. Interviews were recorded and audio transcripts were anonymized prior to analysis.\u003c/p\u003e\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e\u003ch2\u003eData analysis\u003c/h2\u003e\u003cp\u003eWe took a deductive-inductive approach and analysed the transcripts and videos using thematic analysis (\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e). Videos were included in the analysis process as they revealed key non-textual data such as where participants clicked or when they paused to search for information (\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e). Briefly, we familiarized ourselves with the transcripts and videos through repeated reading and viewing. We developed an initial codebook based on the Cognitive Theory of Multimedia Learning (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e). Initial analysis suggested that generative processing and its link to productive struggle was the most relevant for the analysis. Therefore, we developed a final codebook based on productive struggle (Additional file 2, (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e)). Microsoft Word and Excel were used for data management and coding. Discrepancies between coders were resolved through discussion and mutual consensus.\u003c/p\u003e\u003c/div\u003e\n\u003ch3\u003eReflexivity\u003c/h3\u003e\n\u003cp\u003eThroughout this qualitative research process, we were cognisant of how our backgrounds might affect our relationship with the data. KT was, at the time of data collection, a year 4 medical student who approached the project more from a learner\u0026rsquo;s perspective. HSP is a medical educator and researcher with a background in molecular biology, whose views may have been more aligned with that of faculty. Together, we provided both \u0026ldquo;insider\u0026rdquo; and \u0026ldquo;outsider\u0026rdquo; views of how teaching materials are experienced by students.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eA total of ten interviews were conducted with five Year 2 students. Interview lengths ranged from 24\u0026ndash;49 min (median 38 min, total 380 min). Participants have been anonymised (P1-5) followed by whether the interview was based on the PDF or platform version of the manual. We report verbatim quotes with ellipses (\u0026hellip;) to indicate where less relevant text has been cut out and square brackets to indicate any non-verbatim explanatory text.\u003c/p\u003e\u003cp\u003eWe identified three themes centred around non-struggle, generative struggle and non-generative struggle.\u003c/p\u003e\n\u003ch3\u003eNon-struggle leads to low engagement\u003c/h3\u003e\n\u003cp\u003eFor both the PDF and platform-based format, we observed that participants tended to skim through most of the learning material without properly engaging with it.\u003c/p\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;I usually just skim because I, I don't understand the first part, usually like the background and all the equations, so I just skip to the what we're supposed to do the next day part\u0026rdquo;\u003c/em\u003e (P3, PDF)\u003c/p\u003e\u003cp\u003eWhile participants felt the platform-based manual was \u0026ldquo;more reader-friendly\u0026rdquo;, \u0026ldquo;clearer\u0026rdquo; and \u0026ldquo;very structured\u0026rdquo;, the reduced effort of reading did not seem to improve their engagement with the content, with most participants scrolling quickly across the pages with few pauses or verbalization of thoughts. In fact, the use of formatting to signpost important points, while enhancing clarity, sometimes led to participants disregarding content that was not formatted in that way.\u003c/p\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;OK, I guess the other two aren\u0026rsquo;t that important because there's no bolded words\u0026rdquo;\u003c/em\u003e (P3, platform)\u003c/p\u003e\n\u003ch3\u003eGenerative struggle enhances engagement\u003c/h3\u003e\n\u003cp\u003eIn our first round of coding focusing on cognitive load, we found that participants generally preferred the platform-based manual over the PDF manual because it reduced perceived cognitive effort. However, we realized that moments of deeper engagement with the learning material occurred at points that suggested the participants were experiencing higher cognitive load, for example when they paused to think, expressed uncertainty by questioning themselves or navigated to previously read material to check their understanding. This was most evident when the participants attempted the quizzes in the platform-based manual. There was increased verbalization of their thought processes, wherein they expressed doubt in their current understanding and re-read content more slowly to fill in the gaps in their knowledge.\u003c/p\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;I think, then restricted should be lower because for obstruction, I think it\u0026rsquo;s, wait, wait no, wrong. I think should be this because restriction is when you inhale and exhale with difficulty while obstruction is only when you exhale there\u0026rsquo;s difficulty, so I think restriction should have lower vital capacity, then yeah, I think this should be normal, obstructed and treated\u0026rdquo;\u003c/em\u003e (P2, platform)\u003c/p\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;for instance, right, just now I did capacity wrong, so I can go back and refer to what inspiration capacity is\u0026rdquo;\u003c/em\u003e (P4, platform)\u003c/p\u003e\u003cp\u003eParticipants appreciated the fact that attempting the quiz made them think and check their understanding of the content.\u003c/p\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;I feel like it made me, like, have to think on my own, you know how people talk about, like, passive recall active recall, like this actually helped me to make sure that I am like my mind is engaged when I'm doing this and I know what they're talking about\u0026rdquo;\u003c/em\u003e (P5, platform)\u003c/p\u003e\u003cp\u003eWhile infrequent, there were also instances where participants reading the PDF manual appeared to struggle with the content, which led them to process it more deeply.\u003c/p\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;The resulting drop in pressure is measured by using differential pressure trends. Oh, don't really know what that means\u0026hellip; [mumbles while reading the content]. Oh, OK, can be used to confirm a diagnosis and a convenient means of monitoring progress\u0026rdquo;\u003c/em\u003e (P5, PDF)\u003c/p\u003e\u003cp\u003eOverall, we found that participants tended to exhibit signs of engagement with the content when increased cognitive load caused them to pause to think, be it due to a quiz or their own realization of knowledge gaps, rather than skimming through the manual quickly. Therefore, we considered this to be a kind of struggle that led to generative processing.\u003c/p\u003e\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e\u003ch2\u003eNon-generative struggle leads to disengagement\u003c/h2\u003e\u003cp\u003eHowever, increased cognitive effort could also cause participants to avoid reading parts of the learning material. This effort could be due to the perceived complexity of the content or challenges in navigating through the manual.\u003c/p\u003e\u003cp\u003e\u003cem\u003e\u0026ldquo;Maybe I skip through something like P\u003c/em\u003e\u003csub\u003e\u003cem\u003eO2\u003c/em\u003e\u003c/sub\u003e \u003cem\u003ebecause I don\u0026rsquo;t really understand what\u0026rsquo;s P\u003c/em\u003e\u003csub\u003e\u003cem\u003eO2\u003c/em\u003e\u003c/sub\u003e, \u003cem\u003emaybe I read too fast just now\u0026rdquo;\u003c/em\u003e (P1, PDF)\u003c/p\u003e\u003cp\u003eFurther, participants sometimes missed key information due to challenges faced in navigating the manual.\u003c/p\u003e\u003cp\u003e\u003cem\u003e[Participant missed reading large section of text due to clicking on hyperlink earlier] Interviewer: \"Are you able to go back to go, will you be able to go back to those steps?\" P5: \"Oh my gosh, yeah, I just realised I missed this. This one right? OK. Teaching methods. Yes.\u0026rdquo;\u003c/em\u003e (P5, PDF)\u003c/p\u003e\u003cp\u003eWhile participants generally found the platform-based manual easier to navigate, there were still issues with transiting between pages and locating the correct icons to click.\u003c/p\u003e\u003cp\u003eThus, while excessive complexity of content (intrinsic load) and instructional design flaws (extrinsic load) appeared to cause increased cognitive load that made participants pause to think, this did not enhance understanding of content and in some cases, led to disengagement from the content. Therefore, we considered this to be a kind of struggle that was non-generative.\u003c/p\u003e\u003c/div\u003e"},{"header":"Discussion","content":"\u003cdiv id=\"Sec13\" class=\"Section2\"\u003e\u003ch2\u003eSummary of findings\u003c/h2\u003e\u003cp\u003eOverall, our findings revealed two types of points of struggle. Firstly, design flaws and excessively complex content caused excess cognitive load that was counter-productive and caused participants to skip or miss information. Secondly, quizzes or engagement in understanding a challenging piece of information increased cognitive effort but also led to deeper engagement with the material. The affordances of the platform-based manual reduced cognitive load by making navigation of content easier and chunking complex content. However, some cognitive load was still required to drive generative processing and higher order learning.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e\u003ch2\u003eComparison with the literature\u003c/h2\u003e\u003cp\u003eCognitive load theory postulates that learning is less effective if cognitive load exceeds the learner\u0026rsquo;s working memory (reviewed in (\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e)). The three types of cognitive load are extraneous load, intrinsic load and germane load. Extraneous load utilizes working memory but does not contribute to learning. For instance, visual overload may result from excessive text that is presented in a disorganized manner and requires the learner to search for information and piece it together. In our study, this was exemplified in the PDF version of the manual when participants had to read through copious amounts of text and were confused by hyperlinks that brought them to another part of the PDF, such that they lost track of where they had read till. Intrinsic load is caused by the inherent complexity of the task. It can be managed by, for example, chunking complex content into smaller pieces, as was the case for the platform version of the manual.\u003c/p\u003e\u003cp\u003eThe design of digital education tools, especially when based on student preferences for lower effort when learning, tends to focus on the reduction of extraneous and intrinsic load through better interface design and structuring of content. However, effective learning in which information is assimilated into long term memory is dependent on germane load, which requires deliberate effort and concentration from the learner (\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e). In other words, learners must be given appropriately challenging tasks, i.e. DDs, that activate their prior knowledge and engage them in exploration to complete the task, so as to develop conceptual knowledge and transfer through productive struggle (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e). In our study, this tended to occur when participants were presented with quizzes that engaged their attention and activated retrieval practice. Through verbalizing their thoughts and revisiting the learning material, they were able to process the content in a deeper and more integrated manner.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec15\" class=\"Section2\"\u003e\u003ch2\u003eProposed cognitive model of desirable difficulties in digital learning\u003c/h2\u003e\u003cp\u003eBased on the above, we propose the following model of how DD shapes learner engagement in digital learning:\u003c/p\u003e\u003cp\u003eIn the absence of DD and low cognitive load, learners tend to learn superficially and remain disengaged from the material.\u003c/p\u003e\u003cp\u003eHigh cognitive load primarily due to extraneous load, rather than germane load introduced by DD, increases cognitive effort without contributing to learning. This results in non-generative struggle, which may in turn lead to frustration and disengagement as well.\u003c/p\u003e\u003cp\u003eHigh (but not excessive) cognitive load resulting from DDs causes generative struggle in learners. Increased cognitive effort is directed to higher-order thinking and more effective learning.\u003c/p\u003e\u003cp\u003eThe above points are summarized in Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e.\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eRelationship between cognitive load and generative struggle\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"3\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eLack of DD\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003ePresence of DD\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eLow cognitive load\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eDisengagement\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eHigh cognitive load\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eNon-generative struggle\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eGenerative struggle\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003eDesign factors that may contribute to high cognitive load through extraneous load (which should be minimized) include excessive text, spatially disorganized content, unclear formatting and redundant information. Design factors that increase germane load (which should be enhanced) include incorporation of quizzes and other interactive elements that activate retrieval practice. Ideally, quizzes should not just give direct answers but instead provide elaborated feedback to prompt further generative processing in learners (\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eIn our study, we have focused on design factors. Learner factors such as prior level of knowledge, motivation and learning approaches also affect engagement with DD (\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e) and warrant further investigation on how they fit into this model.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec16\" class=\"Section2\"\u003e\u003ch2\u003eImplications for practice and research\u003c/h2\u003e\u003cp\u003eWith the increasing adoption of digital tools in health professions education, there is great potential for utilizing digital affordances to guide learners in analysing, applying and transferring knowledge to achieve higher-order learning outcomes. DD strategies such as retrieval practice, spaced practice and interleaved practice can easily be applied through quizzes and practice tests on digital platforms (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e). The timing and type of feedback on quiz or test performance can be optimized through adaptive systems to enhance long-term retention and deeper learning (\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e). Our work adds to the literature by highlighting the importance of incorporating design elements that introduce DDs, create productive struggle and promote generative processing for deeper learning.\u003c/p\u003e\u003cp\u003eIt is important to raise awareness of DD and productive struggle principles not only amongst medical educators, but medical students as well. Students often fall prey to the \u0026ldquo;fluency illusion\u0026rdquo;, wherein they prefer passive learning strategies such as rereading or highlighting because such strategies feel easier and aid short-term memory, which creates overconfidence in their ability to retain knowledge (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e). Conversely, students tend to reject strategies such as retrieval practice and delayed feedback, despite the robust empirical evidence of their effectiveness, because they require more effort and generate struggle (\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e). Thus, the conventional practice of evaluating digital (and other) educational tools based on institution-wide surveys of student preferences may not provide accurate information on their actual effectiveness.\u003c/p\u003e\u003cp\u003eInstead, evaluation of digital education tools needs to move away from student ratings or tests of short-term retention and instead explore whether and how these tools promote higher-order thinking. This may require more individualised, in-depth studies using methods such as think-aloud interviews to reveal cognitive processes occurring in learners. Further, motivational strategies such as providing an appropriate level of challenge, learning choices and timely feedback can increase students\u0026rsquo; acceptance of DD and other effective learning strategies (\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e). Here, the potential for digital tools to be personalised and flexible, especially coupled with the new affordances introduced by genAI for adaptive learning (\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e, \u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e), can be useful in developing customised learning resources that match the abilities and preferences of students and therefore enhance deeper learning.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec17\" class=\"Section2\"\u003e\u003ch2\u003eLimitations\u003c/h2\u003e\u003cp\u003e As an exploratory study, our sample size was relatively small. Still, we have confidence in our conclusions as the research question was focused, the target group was specific, the study was grounded in theory and the interview data was rich (\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e). Further, a previous study has found that 80% of findings can be detected with just four or five think-aloud participants (\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e).\u003c/p\u003e\u003cp\u003e The process of verbalising thoughts in a think-aloud protocol increases cognitive load for participants, which may have affected how they interacted with the material. Nevertheless, the material used in the think-aloud session was designed for year 1 students and therefore should not have placed excessive cognitive strain on the year 2 participants.\u003c/p\u003e\u003c/div\u003e"},{"header":"Conclusion","content":"\u003cp\u003eWe propose a model for how the introduction of DDs into the design of digital learning platforms can enhance productive struggle, leading to generative processing that enhances learning. In doing so, we identify design factors that may affect disengagement or engagement with the learning material. We also demonstrate how think-aloud interviews can provide insight into the cognitive processes of digital learning platform users to investigate if and how higher-order thinking occurs. Future design of digital learning tools should aim to incorporate DDs with an appropriate level of challenge to enhance generative struggle and therefore make learning more effective.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cp\u003eDD \u0026ndash; desirable difficulties\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cu\u003eEthics approval and consent to participate\u003c/u\u003e\u003c/p\u003e\n\u003cp\u003eEthical approval was obtained from the Nanyang Technological University Institutional Review Board (IRB-2021-1070).\u0026nbsp;Freely-given informed consent was obtained from all participants.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eConsent for publication\u003c/u\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable.\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eAvailability of data and materials\u003c/u\u003e\u003c/p\u003e\n\u003cp\u003eThe datasets used during the current study are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eCompeting interests\u003c/u\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eAuthors’ contributions\u003c/u\u003e\u003c/p\u003e\n\u003cp\u003eKT designed the study, conducted the interviews and performed the analysis. SPH designed the study, performed the analysis and wrote the manuscript. All authors read and approved of the final manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cu\u003eAcknowledgements\u003c/u\u003e\u003c/p\u003e\n\u003cp\u003eWe would like to thank Darren Lim for technical support and access to learning materials used in the data collection.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003e\u003cu\u003eFunding declaration\u003c/u\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eFunding was provided by the Nanyang Technological University Edex grant.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003ePei L, Wu H. Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis. Medical Education Online. 2019 Jan 1;24(1):1666538. \u003c/li\u003e\n\u003cli\u003eRees CE, Nguyen VNB, Foo J, Edouard V, Maloney S, Palermo C. Balancing the effectiveness and cost of online education: A preliminary realist economic evaluation. Medical Teacher. 2022 Sep 2;44(9):977\u0026ndash;85. \u003c/li\u003e\n\u003cli\u003eMeinert E, Eerens J, Banks C, Maloney S, Rivers G, Ilic D, et al. Exploring the Cost of eLearning in Health Professions Education: Scoping Review. JMIR Med Educ. 2021 Mar 11;7(1):e13681. \u003c/li\u003e\n\u003cli\u003eHan SP, Ng O. Teaching touch with technology: Realism and pedagogical intent in digital simulation. Medical Education. 2025 Aug;59(8):792-4. \u003c/li\u003e\n\u003cli\u003eMcGaghie WC, Barsuk JH, Wayne DB, Issenberg SB. Powerful medical education improves health care quality and return on investment. Medical Teacher. 2024 Jan 2;46(1):46\u0026ndash;58. \u003c/li\u003e\n\u003cli\u003eNelson A, Eliasz KL. Desirable Difficulty: Theory and application of intentionally challenging learning. Medical Education. 2023 Feb;57(2):123\u0026ndash;30. \u003c/li\u003e\n\u003cli\u003eBiwer F, Egbrink MGAO, Aalten P, De Bruin ABH. Fostering effective learning strategies in higher education\u0026mdash;A mixed-methods study. Journal of Applied Research in Memory and Cognition. 2020 Jun;9(2):186\u0026ndash;203. \u003c/li\u003e\n\u003cli\u003eKapur M. Examining Productive Failure, Productive Success, Unproductive Failure, and Unproductive Success in Learning. Educational Psychologist. 2016 Apr 2;51(2):289\u0026ndash;99. \u003c/li\u003e\n\u003cli\u003eYoung JR, Bevan D, Sanders M. How Productive is the Productive Struggle? Lessons Learned from a Scoping Review. IJEMST. 2023 Nov 23;12(2):470\u0026ndash;95. \u003c/li\u003e\n\u003cli\u003eGuadagnoli M, Morin M, Dubrowski A. The application of the challenge point framework in medical education. Medical Education. 2012 May;46(5):447\u0026ndash;53. \u003c/li\u003e\n\u003cli\u003eGuadagnoli MA, Lee TD. Challenge Point: A Framework for Conceptualizing the Effects of Various Practice Conditions in Motor Learning. Journal of Motor Behavior. 2004 Jul;36(2):212\u0026ndash;24. \u003c/li\u003e\n\u003cli\u003ePaas F, Sweller J. Implications of Cognitive Load Theory for Multimedia Learning. In: Mayer RE, editor. The Cambridge Handbook of Multimedia Learning. 2nd ed. Cambridge University Press; 2014. p. 27\u0026ndash;42. \u003c/li\u003e\n\u003cli\u003eMayer RE. Cognitive Theory of Multimedia Learning. In: Mayer RE, editor. The Cambridge Handbook of Multimedia Learning. 2nd ed. Cambridge University Press; 2014. p. 43\u0026ndash;71. \u003c/li\u003e\n\u003cli\u003eVan Merrienboer J, Sweller J. Cognitive load theory in health professional education: design principles and strategies. Medical Education. 2010 Jan;44(1):85\u0026ndash;93. \u003c/li\u003e\n\u003cli\u003eYoung JQ, Van Merrienboer J, Durning S, Ten Cate O. Cognitive Load Theory: Implications for medical education: AMEE Guide No. 86. Medical Teacher. 2014 May;36(5):371\u0026ndash;84. \u003c/li\u003e\n\u003cli\u003eDobson JL, Linderholm T. The effect of selected \u0026ldquo;desirable difficulties\u0026rdquo; on the ability to recall anatomy information. Anatomical Sciences Ed. 2015 Sep;8(5):395\u0026ndash;403. \u003c/li\u003e\n\u003cli\u003eOlmsted-Hawala EL, Murphy ED, Hawala S, Ashenfelter KT. Think-aloud protocols: a comparison of three think-aloud protocols for use in testing data-dissemination web sites for usability. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Atlanta Georgia USA: ACM; 2010. p. 2381\u0026ndash;90. \u003c/li\u003e\n\u003cli\u003eWolcott MD, Lobczowski NG. Using cognitive interviews and think-aloud protocols to understand thought processes. Currents in Pharmacy Teaching and Learning. 2021 Feb;13(2):181\u0026ndash;8. \u003c/li\u003e\n\u003cli\u003eEveland WP, Dunwoody S. Examining Information Processing on the World Wide Web Using Think Aloud Protocols. Media Psychology. 2000 May;2(3):219\u0026ndash;44. \u003c/li\u003e\n\u003cli\u003eJaspers M, Steen T, Bos C, Geenen M. The think aloud method: a guide to user interface design. International Journal of Medical Informatics. 2004 Nov;73(11\u0026ndash;12):781\u0026ndash;95. \u003c/li\u003e\n\u003cli\u003eBoren T, Ramey J. Thinking aloud: reconciling theory and practice. IEEE Trans Profess Commun. 2000;43(3):261\u0026ndash;78. \u003c/li\u003e\n\u003cli\u003eBraun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006 Jan;3(2):77\u0026ndash;101. \u003c/li\u003e\n\u003cli\u003eNoushad B, Van Gerven PWM, De Bruin ABH. Twelve tips for applying the think-aloud method to capture cognitive processes. Medical Teacher. 2024 Jul 2;46(7):892\u0026ndash;7. \u003c/li\u003e\n\u003cli\u003eAttali Y, Van Der Kleij F. Effects of feedback elaboration and feedback timing during computer-based practice in mathematics problem solving. Computers \u0026amp; Education. 2017 Jul;110:154\u0026ndash;69. \u003c/li\u003e\n\u003cli\u003eDe Bruin ABH, Biwer F, Hui L, Onan E, David L, Wiradhany W. Worth the Effort: the Start and Stick to Desirable Difficulties (S2D2) Framework. Educ Psychol Rev. 2023 Jun;35(2):41. https://doi.org/10.1007/s10648-023-09766-w\u003c/li\u003e\n\u003cli\u003eHui L, De Bruin ABH, Donkers J, Van Merri\u0026euml;nboer JJG. Why students do (or do not) choose retrieval practice: Their perceptions of mental effort during task performance matter. Applied Cognitive Psychology. 2022 Mar;36(2):433\u0026ndash;44. \u003c/li\u003e\n\u003cli\u003eMullet HG, Butler AC, Verdin B, Von Borries R, Marsh EJ. Delaying feedback promotes transfer of knowledge despite student preferences to receive feedback immediately. Journal of Applied Research in Memory and Cognition. 2014 Sep;3(3):222\u0026ndash;9. \u003c/li\u003e\n\u003cli\u003eHui L, De Bruin ABH, Donkers J, Van Merri\u0026euml;nboer JJG. Does Individual Performance Feedback Increase the Use of Retrieval Practice? Educ Psychol Rev. 2021 Dec;33(4):1835\u0026ndash;57. \u003c/li\u003e\n\u003cli\u003eZepeda CD, Martin RS, Butler AC. Motivational strategies to engage learners in desirable difficulties. Journal of Applied Research in Memory and Cognition. 2020 Dec;9(4):468\u0026ndash;74. \u003c/li\u003e\n\u003cli\u003eMasters K, Herrmann-Werner A, Festl-Wietek T, Taylor D. Preparing for Artificial General Intelligence (AGI) in Health Professions Education: AMEE Guide No. 172. Medical Teacher. 2024 Oct 2;46(10):1258\u0026ndash;71. \u003c/li\u003e\n\u003cli\u003eR\u0026oslash;e Y, Wojniusz S, Bjerke AH. The Digital Transformation of Higher Education Teaching: Four Pedagogical Prescriptions to Move Active Learning Pedagogy Forward. Front Educ. 2022 Jan 14;6:784701. \u003c/li\u003e\n\u003cli\u003eMalterud K, Siersma VD, Guassora AD. Sample Size in Qualitative Interview Studies: Guided by Information Power. Qual Health Res. 2016 Nov;26(13):1753\u0026ndash;60. \u003c/li\u003e\n\u003cli\u003eVirzi RA. Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? Hum Factors. 1992 Aug;34(4):457\u0026ndash;68. \u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Digital learning, educational technology, desirable difficulties, productive struggle, cognitive load theory","lastPublishedDoi":"10.21203/rs.3.rs-7421926/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7421926/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e\u003cp\u003eThe use of digital learning platforms is becoming increasingly common in health professions education, but there are few studies evaluating their effectiveness based on cognitive science principles. Here, we draw upon well-established learning concepts grounded in cognitive science \u0026ndash; desirable difficulties, productive struggle and cognitive load theory \u0026ndash; to explore whether and how digital learning platforms can enhance learning. Our study aims were to i) identify points of struggle when participants processed learning material on a digital platform compared to traditional PDF learning materials and ii) explore how design factors influenced their subsequent responses.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e\u003cp\u003eUsing in-depth think-aloud interviews, we compared how medical student participants engaged with learning material in traditional PDF format versus similar content on an online learning platform (five participants, two interviews each for a total of ten interviews). Participants were instructed to navigate the learning material as they would in their usual practice while verbalizing their thoughts during the process. Interviews were conducted on Zoom and video-recorded. Transcripts and videos were analysed using thematic analysis.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e\u003cp\u003eWe identified three themes centred around non-struggle, generative struggle and non-generative struggle. In the absence of struggle, learners tended to learn superficially and remained disengaged from the material. Desirable difficulties, such as in the form of online quizzes, enhanced learning through introducing points of struggle that led to deeper processing of information, which we term generative struggle. However, struggle that resulted from increased extraneous load due to flawed design was counter-productive and led to disengagement, which we term non-generative struggle.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e\u003cp\u003eBased on our work, we propose a model for how increased cognitive load due to desirable difficulties promotes generative processing and greater engagement with the learning material. This work can guide the future design and evaluation of digital learning platforms for more effective learning based on cognitive science principles.\u003c/p\u003e","manuscriptTitle":"Incorporating Desirable Difficulties into the design of digital learning: A think-aloud study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-17 18:02:22","doi":"10.21203/rs.3.rs-7421926/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorInvitedReview","content":"","date":"2025-10-15T01:06:07+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"226542947249175623076612913900618493133","date":"2025-10-07T14:26:00+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"251343023064772037928348563791443899510","date":"2025-10-06T13:54:09+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-10-06T07:21:09+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-09-10T04:53:46+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-09-09T07:15:31+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-09-09T07:14:19+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Education","date":"2025-08-21T03:51:16+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"7521a5d9-29b9-44cc-8303-7c59522e8d43","owner":[],"postedDate":"October 17th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-10-17T18:02:22+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-17 18:02:22","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7421926","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7421926","identity":"rs-7421926","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}