Closing the Learning Analytics Loop through Explanatory Dashboard Design

Closing the Learning Analytics Loop through Explanatory Dashboard Design | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Closing the Learning Analytics Loop through Explanatory Dashboard Design Sonja Klein, Charlott Sellberg This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7408850/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract This study contributes to the growing body of research on Human-Centred Learning Analytics (HCLA) by exploring how to close the feedback loop between performance and reflection in simulation-based professional learning. Specifically, we investigate how student-facing learning analytics dashboards (LADs) can be designed to support sensemaking through contextualized data presentation, and what kinds of feedback and narrative elements enhance students’ reflective engagement with LADs. Using user-centred design methods and Educational Data Storytelling (EDS), we developed a prototype that integrates HCLA principles with narrative feedback aligned with the educational organization of simulation-based training. The findings demonstrate how explanatory, student-facing dashboards can scaffold interpretation and promote reflection in simulation-based training. The study highlights the importance of moving beyond descriptive analytics toward explanatory designs that actively support student engagement. Future work should involve iterative testing with students and explore how such dashboards can be customized for simulations in different safety-critical domains. Educational Philosophy and Theory Human-Centred Learning Analytics (HCLA) Learning Analytics Dashboard (LAD) Educational Data Storytelling (EDS) Simulation-based training Professional learning Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 1. Introduction In recent years, Learning Analytics (LA) has gained momentum across educational settings, spurring interest in how data-driven tools can meaningfully enhance student learning (Alfredo et al., 2024). A central strategy in this effort has been the use of Learning Analytics Dashboards (LADs), which visualize learners’ data to promote reflection, self-regulation, and awareness (Paulsen & Lindsay, 2024). While LADs hold significant promise, their pedagogical impact remains limited, particularly in complex, open-ended learning environments such as simulation-based training (Buckingham Shum et al., 2019). One recurring critique is that LADs often present decontextualized or overly generic data, which students may struggle to interpret or connect to their own learning processes (Masiello et al., 2024). This challenge is particularly pressing in simulation-based collaborative learning, where learning how to think and act as a professional is highly situated and embodied, shaped by the social and material contingencies of specific situations (Sellberg & Sharma, 2024). Multi-Modal Learning Analytics (MMLA) has been identified to address this challenge by capturing and integrating diverse data modalities, such as speech, gesture, gaze, and analogue tool use, that reflect the embodied and interactional nature of professional learning. By moving beyond clickstream data to encompass the embodied and communicative dimensions of simulation-based activity, MMLA offers a foundation for designing dashboards that can represent learning in these environments and support students’ reflective engagement (Yan et al., 2024). This paper addresses the challenge of supporting students’ sensemaking of LADs in simulation-based maritime training. Drawing on a Human-Centred Learning Analytics (HCLA) perspective (Buckingham Shum et al. 2019), we argue that dashboards should not merely visualize performance metrics but be embedded within pedagogically meaningful structures. To this end, we build on the concept of Educational Data Storytelling (EDS), which emphasizes the need to contextualize data through explanatory features aligned with the learning design (Echeverria et al. 2018). Rather than assuming students can independently make sense of complex dashboard data, our approach advocates for the co-design of LADs that reflect the temporal and pedagogical phases of simulation training, specifically, the sequence of briefing, scenario execution, and debriefing commonly used in maritime education. The study contributes to the growing body of HCLA research by examining how the feedback loop between performance and reflection can be more effectively closed. Specifically, we ask a) how student-facing explanatory dashboards can be designed to support sensemaking through contextualized data presentation, and b) what types of feedback and narrative elements foster reflective engagement in simulation-based learning environments. The design case examined in this study is a simulation-based maritime communication course within a one-year maritime officer program at a Scandinavian university. The course focuses on the Global Maritime Distress and Safety System (GMDSS), where students are trained to use standardized phraseology and radio protocols in safety-critical scenarios. This setting offers a particularly productive context for exploring explanatory and student-facing learning analytics dashboards for two main reasons. First, the maritime domain’s reliance on highly structured communication protocols makes it amenable to automated analysis and feedback. Second, the pedagogical structure of the course, organized into briefing, scenario, and debriefing phases aligns well with narrative-based feedback models such as EDS, allowing dashboard elements to be integrated into existing pedagogical activities. Through participatory design methods (Bannon & Ehn, 2012), we explored how explanatory LAD features, such as feedback messages, visual scaffolds, and narrative cues, can support students in interpreting their own performance and engaging in self-reflection. By integrating EDS with a user-centred design approach, the study offers both conceptual and practical contributions. Conceptually, the study expands the use of narrative and feedback design within student-facing LADs in simulation-based professional training. Practically, it presents tangible, co-developed design features that illustrate the value of aligning learning analytics with pedagogical practice in simulation-based education. The remainder of the article is structured as follows. Section 2 reviews relevant literature on LA, MMLA, LADs, and design principles for explanatory, student-facing tools. Section 3 outlines the theoretical framing, combining HCLA and EDS. Section 4 describes the design case and methodology, including ethnographic observations, participatory design workshops, and prototype development. Section 5 presents the empirical findings, detailing instructors’ feedback practices, co-designed feedback structures, and the resulting LAD prototype. Section 6 discusses the implications of the findings for learning analytics design in professional education, and Section 7 concludes the study. 2. Background The background section provides an overview of LA and MMLA (Section 2.1 ), as well as LADs, focusing on the challenges involved in designing student-facing LADs that provide meaningful feedback and facilitating self-reflection amongst students (Section 2.2 ). Lastly, existing design guidelines for LADs are outlined in Section 2.3 . 2.1 Learning Analytics and Multi-Modal Learning Analytics Learning Analytics (LA) has emerged as a prominent research field aimed at improving learning and teaching through data-driven insights (Alfredo et al. 2024 ). Defined in 2011 at the first International Conference on Learning Analytics and Knowledge (LAK), LA involves “ the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs ” (SOLAR, 2025 ). As a field, LA emphasizes a holistic systems perspective and has predominantly focused on leveraging computational methods to explore and support social and educational processes (Baker & Siemens, 2014 ). A central debate within LA concerns what constitutes meaningful data and how learning should be conceptualized. Early LA research relied heavily on easily accessible online activity data, often grounded in behaviourist assumptions (Verbert et al., 2013 ). However, this focus risks producing a “streetlight effect,” wherein attention is restricted to measurable activities rather than to contexts where LA could offer real pedagogical value (Ochoa, 2022 ). Others, like Blikstein and Worsley ( 2016 ), have highlighted the challenges of applying reductive data models in constructivist or socio-cultural learning contexts, which often feature open-ended and complex learning processes. To move beyond these limitations, Ochoa ( 2022 ) calls for the integration of diverse data streams and context-specific behavioural markers, offering a pathway to more meaningful LA systems. Addressing this need, Multi-Modal Learning Analytics (MMLA) has emerged as a subfield of LA, focused on expanding LA into collaborative and open-ended educational settings. As defined by Worsley et al. (2016), MMLA “ utilizes and triangulates among non-traditional as well as traditional forms of data in order to characterize or model student learning in complex learning environments ” (p. 1346). First introduced by Blikstein (2013), MMLA responds to the limitations of earlier LA approaches by recognizing the inherently multimodal nature of learning, particularly within student-centred and open-ended pedagogies. In such settings, MMLA incorporates a variety of sensor technologies, including video, audio, bio signals, and motion capture, to record and analyse aspects of learning previously difficult to observe. These data are processed into meaningful modalities depending on the learning context, for example, through speech-to-text transcription or gesture recognition (Ochoa, 2022 ). This allows for a holistic analysis of learning, ultimately making formative feedback and assessment more relevant and actionable (Blikstein & Worsley, 2016 ). Moreover, MMLA has been positioned as a bridge between Learning Analytics and Learning Design (LD), ensuring that analytics align with pedagogical intentions (Sharma & Giannakos, 2020 ). For instance, recent research in simulation-based education shows that when MMLA tools are meaningfully integrated into instructional practice, they can offer educators valuable, context-sensitive insights (Echeverria et al., 2018 ). 2.2. Closing the Loop through Learning Analytics Dashboard Design Learning Analytics Dashboards (LADs) are central interfaces through which insights from learning analytics (LA) are returned to students, educators, and other stakeholders. As visualization tools, LADs present traces of learning activity intending to foster awareness, reflection, sensemaking, and goal setting (Verbert et al., 2013 ). Ideally, LADs are designed not simply to display data, but to make that data pedagogically actionable. Paulsen and Lindsay ( 2024 ) identify comparison, awareness, and monitoring as the most common design elements in LADs, often supplemented by features supporting goal-setting, recommendations, and self-assessment. Similarly, Schuhmacher and Ifenthaler (2018) highlight learners’ preference for timely, actionable feedback, suggesting that LADs are most effective when they inform learning in the moment rather than after the activity. LADs typically operate at three levels: descriptive (what happened), diagnostic (why it happened), and predictive (what might happen next or what to do). While descriptive LADs dominate current implementations, diagnostic and predictive versions remain underutilized (Paulsen & Lindsay, 2024 ). In response, researchers have begun to develop explanatory LADs that offer interpretive guidance. Echeverria et al. ( 2018 ), for example, argue for contextualized, task-aligned dashboards, drawing an analogy to car dashboards that selectively present only what is needed for the current driving conditions. Similarly, Worsley et al. ( 2021 ) co-designed a LAD with students for collaborative teamwork, embedding visual cues such as color-coded scripts and interaction timelines based on students’ own criteria for success. These examples point toward situated and learner-centred approaches to LAD design. Conceptually, LADs are often framed within the broader logic of the LA “loop”, a cyclical process where data is collected from learner activity, analysed, and fed back to inform future learning. Clow ( 2012 ) introduced this loop as a model to emphasize that the value of LA lies not in data collection alone, but in how those data lead to meaningful changes in learning behaviour. The loop is only “closed” when the insights delivered through tools like LADs lead to pedagogically meaningful interventions, whether initiated by learners themselves or by educators. Wise et al. ( 2021 ) reinforce this view, noting that LA must ultimately be evaluated in terms of its pedagogical impact. Unless insights are acted upon, LA systems remain inert, regardless of how sophisticated their analytics may be. While Clow ( 2012 ) stresses the importance of student understanding, he also argues that pedagogical value can still be achieved if faculty respond productively to the data. Wong and Li’s ( 2020 ) review illustrates the variety of possible interventions, ranging from efforts to improve retention and academic performance to strategies for fostering engagement and deeper learning. Still, as Wise et al. ( 2021 ) point out, the challenge of “closing the loop” remains a persistent blind spot in both research and practice. This study addresses that gap by focusing on how LADs are not only designed but used, particularly how their structure supports or constrains, learners’ ability to make sense of data and act upon it. In this regard, the pedagogical value of LADs depends not only on their visual or technical sophistication but also on whether students can interpret and respond to the information they present. In the following section, we explore the conditions under which students make sense of LADs and the factors that support or hinder this process. 2.3 Design Guidelines for Learning Analytics Dashboards This section focuses on design strategies relevant to developing explanatory, student-facing Learning Analytics Dashboards (LADs), grounded in critical literature but emphasizing those that directly inform the design space explored here. Central to this synthesis is information visualization, a foundational approach in LAD development aimed at simplifying complex data through cognitive pattern discovery (Klerkx et al., 2017 ). Although this approach might appear to conflict with the view of students as active agents within human-centred design, aligning visualization techniques with learning theory is essential to maximize pedagogical effectiveness (Vieira et al., 2018 ). 2.3.1 Visual Structuring LADs typically leverage visual elements, colour, text formatting, layout, and representations like charts, maps, or flow diagrams to enhance interpretability (Le et al., 2024 ; Masiello et al., 2024 ). Adding contextual cues, such as inferential text or data labels, improves comprehension (Alhadad, 2018 ; Wiley et al., 2024 ). For simulation-based exercises, sequential visualizations like flowcharts help students reflect on decision-making processes within dynamic scenarios (Le et al., 2024 ). While visualizations must be pedagogically driven (Masiello et al., 2024 ), colour use is nuanced: it can direct attention but risks biasing interpretation if not carefully designed (Ahn et al., 2019; Roberts et al., 2017 ). Text formatting and layout techniques like decluttering and chunking support focus and readability (Alhadad, 2018 ; Wiley et al., 2024 ). Interactive elements, filters, links, guided tours, enable deeper data exploration without overwhelming users, with educational experts valuing such hierarchical navigation in LADs (Bodily & Verbert, 2017 ). 2.3.2 Comparisons and Contextualization Grounding LAD data in the learning context enhances relevance, often through links to curricula, learning goals, or assessment items (Wiley et al., 2024 ). Comparisons with personal or class performance indicators are common (Villagrán et al., 2024 ), though student responses vary: while some find peer comparisons motivating, others perceive them as discouraging (Roberts et al., 2017 ). Recommendations derived from data mining offer actionable next steps but often lack transparency, impacting trust and uptake (Bodily & Verbert, 2017 ). When framed with a rationale, as in teacher- and student-facing LADs, such recommendations receive positive feedback (Wiley et al., 2024 ; Villagrán et al., 2024 ). 2.3.3 Interpreting and Narrating Data Data storytelling structures LAD content narratively to foster interpretation and reflection, often following a narrative arc with critical moments and resolutions (Asgari & Hurtut, 2024 ). Data storytelling has shown promise in explanatory LADs for complex data, aiding reflection and interpretation in fields such as nursing education (Echeverria et al., 2023). Layered narrative designs support instructor-led debriefings and increase student accountability (Martinez-Maldonado et al., 2020 ). Summative textual evaluations combining descriptions of students actions and group feedback further assist understanding, with AI-generated summaries emerging as a useful tool (Bodily & Verbert, 2017 ). 2.3.4 Feedback and Reflection Though not always framed as an explicit design strategy, feedback literacy underpins students’ sensemaking of LADs and highlights feedback’s reflective function (Carless, 2019 ; Hattie & Timperley, 2007 ). Effective feedback addresses three key questions: Where am I going? How am I going? Where to next? (feed up, feed back, feed forward). Feedback must align with learning goals and be actionable, contextualized, and matched to the learner’s understanding and task (Hattie & Timperley, 2007 ). Reflection, a critical aspect of feedback uptake, remains underexplored in LAD design, though self-assessment features have been implemented with mixed student engagement (Villagrán et al., 2024 ). In sum, the literature underscores the need for explanatory, learner-centred LAD designs that close the learning analytics loop through integrated visualization, contextualization, narrative, and feedback strategies. The key challenge lies in translating these theoretical strategies into tangible, pedagogically meaningful dashboard features that foster student reflection and sensemaking. 3. Theoretical Framing This study draws on the combined perspectives of Human-Centered Learning Analytics (HCLA) and Educational Data Storytelling (EDS) to inform both the design and evaluation of student-facing dashboards. These frameworks are united in their emphasis on learner agency, pedagogical alignment, and the interpretability of data within authentic educational settings. Human-Centered Learning Analytics (HCLA) has been described as a counter-position to technically driven learning analytics systems, arguing that “it is more sensible to change the tools to suit their users, rather than changing the users to suit the tools” (Buckingham Shum et al., 2019 , p. 5). HCLA foregrounds learners as co-constructors of meaning, rather than passive recipients of feedback, and emphasizes that LA tools must be aligned with the pedagogical and contextual affordances of the learning environment (Dimitriadis et al., 2021 ). Key commitments of HCLA include the agency of stakeholders, integration with learning design, and grounding in educational theory. As emphasized, this approach views LA not just as a set of metrics, but as part of a socio-technical system designed in collaboration with users, particularly students and instructors (Alfredo et al., 2024 ). To complement this perspective, Educational Data Storytelling (EDS) provides a design-oriented framework for embedding learning analytics in narrative structures. Developed as a response to the limitations of static visualizations, EDS scaffolds learners' interpretive process by aligning analytics with pedagogical intent and temporal learning trajectories (Echeverria et al., 2018 ). Rather than offering exploratory dashboards that assume high levels of data literacy, EDS supports explanatory approaches that foreground selected data points and offer clear narrative cues to help students understand what happened, why it mattered, and how they can improve (Fernández-Nieto et al., 2024; Martinez-Maldonado et al., 2020 ). In line with this, EDS is positioned as a “middle ground” between personalized and generic dashboards, offering a balance between individualized insight and general contextual scaffolds (Echeverria et al., 2018 ). By combining HCLA and EDS, this study responds to the epistemological and practical challenges of designing learning analytics tools for professional learning environments. The joint application of these frameworks allows us to reconceptualize dashboards from monitoring tools into narrative spaces that support students’ agency, reflection, and ongoing development. This reconceptualization is particularly relevant in simulation-based maritime training, where the reflective space created by debriefing plays a central role in learning. During debriefings, feedback typically unfolds as a narrative, drawing on domain-specific rules, prior experiences, and imagined future scenarios (Karahalil et al., 2023 ). In this way, debriefings align closely with the epistemic principles of EDS and offer a promising entry point for rethinking dashboard design. To explore this potential in context, we turn now to the setting in which our study was conducted. 4. The Design Process The participatory design process guiding this research follows Spinuzzi’s ( 2005 ) three-stage model: exploration , discovery , and prototyping (Fig. 1 ). The research method will be described in terms of these four interconnected stages. While presented as distinct for analytical purposes, the phases are inherently overlapping. In practice, prototyping occurs iteratively as part of a continuous effort to develop and refine educational designs (Cohen et al., 2011; Bannon & Ehn, 2012 ). Given that this study builds on prior design research, the process should be understood as part of an iterative design trajectory. Before describing the method, an overview of the design case is provided. 4.1 The Design Case The study examines a nine-week course in maritime communication within a one-year maritime officer program at a Scandinavian university, focusing on Global Maritime Distress and Safety Systems (GMDSS). The course aimed to develop students’ ability to communicate using maritime standard communication phrases (SMCP)[1] and to operate GMDSS[2] radio equipment. The course consists of theoretical lectures and simulation-based training. Weekly lectures introduced technical and operational content, including standard phraseology in Swedish and English, followed by mandatory simulator sessions. Simulator sessions took place in a dedicated radio simulator classroom featuring five student cabins and one instructor cabin, each equipped with Very High Frequency (VHF) and Digital Selective Calling (DSC) simulators (Fig. 2 ). These supported the simulation of distress and routine traffic communication across different systems. The pedagogical model followed a briefing-scenario-debriefing structure. Briefings introduced the scenario and equipment, while scenarios involved paired students simulating ship-to-shore communication, though the interaction was conducted individually. Each student enacted scripted procedures tailored to their ship’s identity, communicating with the instructor acting as a coastal radio station. The instructor occasionally paused the simulation to provide immediate feedback, and all sessions concluded with a debriefing that emphasized key learning points and professional relevance. Research ethics were carefully considered throughout the study, concerning privacy, consent, and potential risks to participants. As the simulator sessions reflected routine and non-graded learning activities, the risk of harm was assessed as minimal. All participants were fully informed about the aims and procedures of the study and gave their written consent before data collection. In the participatory design workshop, additional consent was obtained for the use of identifiable student video recordings, which were necessary to preserve the communicative nuances (e.g., gestures, posture, tone of voice) central to the feedback practices under investigation. Data were handled following local ethical guidelines, ensuring confidentiality, secure storage, and restricted access. 4.2 Design Ethnography To explore the situated dynamics of simulator-based learning, two video- and audio-recorded sessions were conducted as part of a design ethnographic study (Crabtree et al. 2013 ). The sessions took place over two days in the GDMSS simulator at a Scandinavian university, each session lasting approximately four hours. Although the same instructor facilitated both sessions, different student groups (8–10 participants) attended. The video and audio recordings were supplemented by the collection of relevant artifacts, including curricula, learning objectives, and instructional materials, to contextualize the observed activities and feedback practices. Access was granted through the lead instructor, and session selection followed his recommendations. Aligned with Spinuzzi’s ( 2005 ) emphasis on early-stage ethnographic exploration, this approach supported an open-ended examination of the enacted learning situation, with a focus on capturing feedback interactions and identifying candidate episodes for a future MMLA system. Field notes were used to support orientation within the audiovisual data corpus. Thematic analysis (Clarke & Braun, 2014 ) guided the post-observational phase. Via critical case sampling (Clark et al., 2021), eight incidents were selected for detailed analysis, covering briefings, in-action feedback, debriefings, and three instances of student activity, based on their relevance to instructional feedback. Transcription was performed using Whisper AI and manually reviewed for accuracy. 4.3 Participatory Design Workshop To triangulate observational findings and involve stakeholders in the design of a LAD with multimodal learning analytics, a participatory design workshop was conducted. The workshop aimed to co-design feedback rules that could later be implemented in the LAD (Echeverria, Martinez-Maldonado, Granda, et al., 2018). Four instructors in Maritime Communication from two European universities, recruited through purposive sampling (Clark et al., 2021) participated. During the 90-minute session, participants viewed three selected student video recordings and discussed the feedback they would normally provide. These sequences were shown unaltered, based on the relevance of body posture, gesture, and tone for interpreting feedback practices. The workshop discussion focused on identifying critical feedback moments and collaboratively exploring how such feedback is typically delivered and could be encoded as rules for automation. The session was transcribed in a denaturalized format (Oliver et al., 2005) using Whisper AI and manually validated. Thematic analysis (Clarke & Braun, 2014 ) was then conducted using NVivo, combining this material with data from the exploratory phase. Coding followed a deductive structure oriented around five W-questions, Who, How, What, Why , and When , resulting in six codes: learning context, learning content, learning intentions, feedback indicators, feedback message, and feedback deliverer. These were synthesized into four broader themes: learning intentions, indicators, timing, and feedback messages. Additionally, nine specific feedback targets emerged, including clarity of speech, number of transmissions, affective state, initial call structure, channel switching, and use of the Push-to-Talk (PTT) button. This analysis provided a granular understanding of how instructors structure feedback, which is essential for designing human-centred feedback in MMLA systems (Dimitriadis et al., 2021 ; Echeverria et al., 2018 ). 4.4 Prototyping Following Lim et al. (2008), prototyping was seen as a method for evolving and exploring design ideas iteratively. The prototypes served as embedded artifacts to express and refine core pedagogical concepts, especially those related to EDS and feedback practices in simulator-based training. To narrow the scope while maintaining relevance, the prototyping focused on the theme of “clarity,” identified as both widely applicable in simulation contexts and particularly suited for enhancement through MMLA. The theme also aligned with student feedback expressing interest in improving the clarity of their speech. This focus helped balance generalizability with pedagogical grounding. The base prototype was developed by digitizing earlier pen-and-paper designs using Figma, integrating prior feedback on usability aspects such as navigation and iconography (Fig. 3 ). A digital format was chosen to allow for interactive features, aligning with the complexity of EDS elements and facilitating meaningful stakeholder evaluation (Lim et al., 2019 ). New design elements were developed through a 20-minute sketching session and further refined in an iterative process informed by earlier student and instructor input. The final interactive prototype included multimodal components such as graphs, audio feedback, reflective prompts, interactive questions, and color-coded transcript videos. The audio feedback was created using AI-generated voice recordings based on synthesized student speech scripts to protect participant privacy and enhance pedagogical richness by combining multiple learning indicators. 5. Results The findings show how instructors’ feedback during simulation activities conveys specific learning intentions that can inform the design of feedback elements in LADs, and illustrate how these learning intentions, alongside the contextual feedback narratives in which they are embedded, can be translated into LAD features that support students’ sensemaking. The results section outlines our empirical findings of feedback practices in GDMSS training (Section 5.1 ), the instructors’ feedback designs (Section 5.2 ), and the prototype design (Section 5.3 ). 5.1 Feedback Practices in GDMSS Training During GMDSS simulation exercises, instructors frequently framed their feedback around students’ use of maritime English, with an emphasis on clarity, linguistic efficiency, and adherence to standard communication protocols. For example, one recurring point of feedback concerned students’ failure to say “ over ” at the end of transmissions, which disrupted turn-taking and led to unnecessary repetitions. Instructors described this breakdown using metaphors, such as throwing a ball, to illustrate the correct turn-taking order. These kinds of feedback interactions reveal not only what students are expected to do but also the intentions that underpin these expectations. This section examines how such feedback practices can inform the development of feedback features in Learning Analytics Dashboards (LADs). By analyzing expected and observed student actions alongside instructor responses, we identify how contextualized, narrative-rich feedback can be translated into LAD elements that support students’ reflective sensemaking. Where relevant, design suggestions from an instructor workshop are included, particularly concerning automated tracking and visualization of communication performance. Each feedback case is structured around the expected student action, the underlying learning intention, the observed action, and the instructor’s response. Where applicable, ideas from the instructor workshop regarding automation and visualization are also presented. Quotes from instructors (I1, I2, etc.) and the observed lecturer (L1) are included to illustrate specific insights while maintaining anonymity. Across the cases, instructors generally described their learning intentions as helping students practice maritime English, use standardized phrases, and become familiar with the GMDSS simulator environment. A central expectation was that students speak maritime English consistently throughout the exercises. One key focus was on the number of transmissions used in communication, which, according to European regulations, should match a targeted number. Deviations were often attributed to unclear turn endings, which instructors flagged as reducing communication efficiency. They expressed interest in LAD features that could visualize where transmission protocols, such as saying “ over ”, were correctly or incorrectly followed. A second learning intention centred on the clarity of speech, defined by tempo, pronunciation, and appropriate pausing between word groups, in line with IALA (2022) recommendations. Positive comments such as “ good tempo ”, “ clear speech ”, or “ good rhythm ” were commonly used, though they tended to be brief and lacked narrative elaboration. In the workshop, instructors proposed including a clarity percentage or intelligibility score in the LAD, emphasizing the operational importance of being understood on the bridge. One instructor suggested that if speech was unintelligible, this alone should count as a feedback metric. Linguistic efficiency emerged as a third key learning intention: students were expected to express messages using the fewest necessary words, avoiding fillers and repetitions. Despite this, instructors often observed filler words and digressions during simulation tasks. Rather than framing these as outright errors, they normalized them, “ It’s normal. It’s there ” (L1), and used these moments to introduce mitigating strategies, such as saying “standby” or releasing the Push-to-Talk button when thinking. During the workshop, instructors suggested including a feature in the LAD that compares the actual number of words used to the ideal phrasing based on the Standard Marine Communication Phrases: “ You were trying to say this, it requires according to the SMCP code four words, and you used 28 ” (I1). Across all examples, instructor feedback, though mostly positive, frequently highlighted recurring issues through metaphors, case examples, or personal experience. These narrative elements not only made the feedback more accessible but also revealed instructional concerns that aligned closely with themes raised in the workshop. Together, these findings suggest that feedback in GMDSS training functions as a narrative practice grounded in situated learning intentions. 5.2 Instructors’ Feedback Designs In the instructor workshop, participants reflected not only on the intentions behind their feedback but also on broader design considerations for integrating feedback into LADs. Overall, the instructors discussed the feasibility of visualizing student performance data, the potential for automating feedback messages, and the role of the LAD in supporting debriefings. A recurring theme in the conversation was the practical limitations of current feedback practices, particularly during more advanced simulations. Instructors emphasized that certain feedback patterns were stable across sessions, making them suitable for automation. “ If you generalize it a little bit…” one instructor explained, “ you always find the same good or bad… or at least a lot of them. So those, you can create as a feedback bank ” (I2). Rather than replacing instructor judgment, such a feedback bank could handle basic, recurrent issues, allowing instructors to focus their attention on more complex, situation-specific matters. This point was reinforced in discussions about workload. “ I don’t want to go into all the independent cabins and like, ‘okay, so look…’, because there’s no time for that ” (I1), one instructor noted, highlighting time constraints during simulations as a barrier to individual feedback. This challenge was especially relevant in relation to communication performance, which instructors described as frequently overlooked during assessments. As one instructor put it, “ As long as you have some kind of communication that is understood, you don’t comment on it so much, because you will be focusing on COLREGs [3] , or whatever it is, right? ” (I1). Here, the potential value of the LAD became particularly evident: instructors envisioned using it as a tool to foreground aspects, like communication clarity, that otherwise risk being neglected. They emphasized its value not as a stand-alone assessment tool, but as a prompt for post-exercise reflection. The LAD, they agreed, would be most useful during debriefings to support structured, data-driven discussions with students. When discussing how feedback data might be presented in the LAD, instructors expressed mixed feelings about the idea of automated interpretation. Some worried about demotivating students: “ What if they just get thumbs down all the time?” (I1). Rather than labeling performance as “ good ” or “ bad ”, instructors advocated for neutral visualizations, such as ratios, percentages, or other forms of quantified data, that could serve as starting points for conversation. Final interpretations, they argued, should be left to instructors and negotiated with students. Still, the instructors recognized the pedagogical value of certain interpretive features, especially if linked to students’ own actions. One instructor was particularly enthusiastic about the potential of integrating audio playback with automatic analysis of speech: “ If you can replay and listen to yourself… that is a pedagogical masterpiece actually! And especially if the computer then [says]: ‘unnecessary,’ ‘not according to SMCP’ ” (I1). Such a feature, they suggested, could make regulatory alignment visible and actionable, combining narrative and data to support learning, a central principle in EDS. During the workshop, instructors also offered concrete ideas for how feedback could be visualized. One proposed using line graphs to show changes over time. Others suggested layering information, using icons or short evaluative phrases, to make complex feedback easier to interpret at a glance. Taken together, the workshop discussions show that instructors view the LAD primarily as a reflective tool to support feedback conversations, rather than a mechanism for judgment or grading. They were optimistic about the possibility of automating context-sensitive feedback messages, particularly those grounded in repeated patterns of student action. At the same time, they expressed caution about overly directive or interpretive LAD features, emphasizing the need for transparency, flexibility, and pedagogical alignment with their instructional goals. 5.3 Prototype Design The LAD prototype was developed to visualize feedback from simulator-based maritime communication training, with an initial design focus on clarity of speech. The base interface provides a session overview, including student role, course ID, date, and curriculum-based exercise title. Each session includes an audio playback of the communication activity and an overview of feedback categories, “ Clarity ”, “ Information Exchange ”, and “ PTT Checks ”, displayed through interactive summary metrics and expandable trend graphs (see Fig. 3 ). In its first iteration, only the “ Clarity ” category was elaborated in detail. This category was subdivided into “ Speed & Pauses ”, “ Filler Words ”, and “ Unnecessary Repetitions ”. Each sub-category is associated with a development graph, a five-session moving average, and trend indicators. Feedback is presented using an “ if-indicator-then-message ” structure: when specific linguistic markers are detected (e.g., filler word use or speech rate deviation), feedback messages are generated to prompt learning insights. Feedback cases were categorized under the broader learning objective of promoting speech aligned with SMCP. For example, speech rated as too fast or lacking pauses is flagged under “ Speed & Pauses ”, while redundant phrasing or filler terms trigger feedback in their respective sub-categories. Table 1 illustrates how specific communicative features were mapped onto LAD categories, alongside the types of feedback responses they triggered (e.g., positive reinforcement, self-reflective prompts, SMCP references, or regulatory comparisons). Table 1 Mapping of Category Clarity with the overall learning intention: “The student speaks clearly according to Standard Maritime Communications and Phrases Regulations.” Feedback Case Analysed Feedback Indicators Analysed Feedback Messages LAD Category Speaking clearly Student takes no breaks Student speaks faster or slower than 120 words per minute Positive feedback (e.g. ”good speed”) Speed & Pauses Using minimum amount of words necessary Use of filler words (e.g. “ehm”) Justification of normalcy Self-reflecting question Proposing coping strategy Referral to SMCP regulations Comparison necessary words/words used Filler Words Repeating words unnecessarily Unnecessary Repetitions 5.1.1 Visual Feedback Design To support longitudinal reflection on performance, interactive line and bar graphs were embedded in the dashboard. These visualizations track individual progress across multiple sessions, with session-specific data accessible via clickable nodes. In response to previous iterations of the prototype, a color-coded background was added in a second design iteration to support intuitive interpretation (Fig. 5 ). A traffic-light colour scheme (green/yellow/red) was used to denote performance zones. For instance, a rate near 120 words per minute was defined as optimal for clarity (green), while a low frequency of fillers indicated effective communication. Although instructors expressed reservations about over-reliance on visual coding, the colour scheme was retained due to its interpretive affordances for students unfamiliar with numerical metrics. 5.1.2 Audio Transcript Playbacks Drawing on insights from earlier participatory design sessions with maritime students, where participants emphasized the importance of accessing and reviewing their simulator data (Harrington et al. 2025 ), the prototype integrates audio transcript playbacks that align line-by-line with students’ audio-recorded communication. These playbacks highlight speech in real time, with color-coded transcripts indicating whether utterances matched or violated expected standards. Within “ Filler Words ” and “ Unnecessary Repetitions ”, system-detected infractions are marked in red, while compliant speech appears in green (Fig. 5 ). To complement the students’ transcripts, model transcripts of correct performance were included. These are presented without colour markings but are accompanied by interpretive feedback to illustrate what students should have said. This dual representation, student output and a regulatory-conforming exemplar, enables comparative reflection and supports corrective learning. 5.1.3 Reflective Feedback and Scaffolded Self-Assessment Info banners provide a textual summary of key data points, including what the student attempted, applicable SMCP regulations, and comparative evaluations. These banners were designed to bridge data representation and pedagogical interpretation, particularly for learners less comfortable navigating charts. They also serve as entry points to more detailed feedback layers. To mimic instructor-facilitated reflection during post-simulation debriefings, the prototype incorporates self-assessment features. These include open-ended prompts and multiple-choice questions embedded within specific categories. For example, in Filler Words, students are first shown an info banner summarizing observed speech behaviour and its misalignment with regulations. Upon expanding this section, they are prompted to reflect on possible coping strategies, with the correct answer withheld until the “show answer” option is clicked. This deliberate delay is designed to scaffold metacognitive engagement. In Speed & Pauses, incidents such as unusually long silences (e.g., a 22-second pause) are flagged, with students asked to select the most plausible explanation from three options. Feedback is delivered via visual outlines (red/green) and is followed by an interpretive explanation, often including metaphors or contextualized recommendations for improvement. These feedback layers collectively aim to foster student agency in interpreting data, integrating instructor-authored feedback cases into an interactive, student-centred learning analytics interface (Harrington et al. 2025 ). 6. Discussion This study set out to explore how reflective feedback features can be designed for LADs in simulator-based training, building on the principles of HCLA and EDS. The aim was to address a noted gap in LA and MMLA research: the need to support students not just with data points, but with pedagogically meaningful feedback that can foster sense-making and self-assessment (Echeverría et al., 2018). Through a prototype focused on maritime radio communication, the study investigated how LADs might visualize and contextualize feedback in a way that resonates with both the structure of simulator-based learning and the social dynamics of feedback practices. The findings suggest that feedback in simulation contexts is highly narrative, situated, and oriented toward regulation-based communicative competence. Instructors’ use of metaphors, case-based examples, and regulatory framings provides a natural bridge to EDS. In particular, the notion of “ throwing the ball ” to describe conversational turn-taking illustrates how instructors already embed narrative logic into their feedback to support students’ understanding. This observation supports the use of LADs not merely as repositories of performance data but as tools that can extend such narratives through carefully designed visualizations and interactions. The incorporation of features like color-coded transcripts, audio playback, and reflective questions demonstrates how LADs can align with and enhance these narrative practices. The findings also reveal how instructors’ existing feedback practices include pedagogical narratives that can be directly leveraged in LAD design. Rather than relying on generic indicators, instructors use situated metaphors, regulatory references, and normalizing framings to help students interpret their communication performance. These narratives, often repeated across student interactions, form a repertoire for context-aware automation, as instructors themselves noted. Their reflections pointed to a dual role for LADs: as tools for individualized reflection and as prompts for instructor-led discussion during debriefings. Importantly, instructors saw value in automating foundational feedback while reserving interpretive authority for more complex or ambiguous cases. This aligns with the broader ambition of explanatory dashboards: not to replace human feedback but to provide meaningful scaffolds (Echeverria et al., 2018 ). Still, tensions remain around how much interpretation should be embedded in LADs. While instructors welcomed features like audio playback and SMCP-based benchmarking, the instructors expressed caution toward prescriptive judgments (e.g., thumbs-down icons), advocating instead for neutral data displays that support shared interpretation. These insights underscore the value of designing LADs that support professional judgment while expanding opportunities for learner reflection, an approach consistent with both HCLA and EDS frameworks. These findings also speak to a central tension in the learning analytics literature: the balance between interpretability and prescription (Buckingham Shum et al., 2022; Ifenthaler & Yau, 2020). While the instructors valued LADs that could automate aspects of feedback delivery, they resisted systems that might fix meanings or evaluations without space for negotiation. This reflects ongoing critiques of dashboards that impose normative models of learning without attending to the contextual, relational, and interpretive dimensions of educational practice. The workshop participants’ interest in integrating narrative metaphors, student audio, and regulatory references illustrates how EDS can be operationalized in practice, not to deliver definitive judgments, but to scaffold students’ and instructors’ shared interpretive work. In this sense, the LAD is not just a feedback tool but a boundary object that mediates between data traces, professional teaching knowledge, and student reflection. Designing for such a role requires not only technical functionality but also humility towards what it means to learn in complex safety-critical domains: the willingness to foreground uncertainty, enable dialogue, and support human sense-making over algorithmic closure (Buckingham Shum et al., 2022; Martínez-Maldonado, 2020). Taken together, the study contributes to a growing body of literature calling for a shift from performance-focused dashboards toward reflective and dialogic tools that extend existing pedagogical practices. Overall, the study demonstrates how LADs can be meaningful not by delivering answers, but by prompting the kinds of questions instructors typically ask, and by giving students the tools to ask them of themselves. 7. Conclusion This study contributes to the field HCLA by demonstrating how carefully designed dashboards can function as meaningful reflective tools in simulation-based professional education. Through the integration of narrative feedback and contextualized data presentation, the prototype reimagines feedback in learning analytics not as abstract metrics, but as part of a coherent narrative structure that supports learners’ sensemaking. The findings underscore the value of aligning feedback with the pedagogical practices of simulation training, enabling retrospective reflection grounded in concrete learning scenarios. In this way, the dashboard serves as a reflective space that promotes deeper engagement with one’s practice and supports ongoing professional development. References Alhadad, S. S. J. (2018). Visualizing Data to Support Judgement, Inference, and Decision Making in Learning Analytics: Insights from Cognitive Psychology and Visualization Science. Journal of Learning Analytics , 5(2), 60–85. https://doi.org/10.18608/jla.2018.52.5 Alfredo, R., Echeverria, V., Jin, Y., Yan, L., Swiecki, Z., Gašević, D., & Martinez-Maldonado, R. (2024). Human-centred learning analytics and AI in education: A systematic literature review. Computers and Education: Artificial Intelligence , 6 , 1–23. https://doi.org/10.1016/j.caeai.2024.100215 Asgari, M., & Hurtut, T. (2024). A Design Language for Prototyping and Storyboarding Data-Driven Stories. Applied Sciences , 14(4), 1387. https://doi.org/10.3390/app14041387 Baker, R., & Siemens, G. (2014). Educational Data Mining and Learning Analytics. In R. K. Sawyer (Ed.), The Cambridge Handbook of the Learning Sciences (2nd ed., pp. 253–272). Cambridge University Press. https://doi.org/10.1017/CBO9781139519526.016 Bannon, L. J., & Ehn, P. (2012). Design Matters in Participatory Design. In: Routledge International Handbook of Participatory Design (1st ed., pp. 37–63). Routledge. https://doi.org/10.4324/9780203108543 Blikstein, P., & Worsley, M. (2016). Multimodal Learning Analytics and Education Data Mining: Using computational technologies to measure complex learning tasks. Journal of Learning Analytics , 3(2), 220–238. https://doi.org/10.18608/jla.2016.32.11 Bodily, R., & Verbert, K. (2017). Trends and issues in student-facing learning analytics reporting systems research. Proceedings of the Seventh International Learning Analytics & Knowledge Conference , 309–318. https://doi.org/10.1145/3027385.3027403 Buckingham Shum, S., Ferguson, R., & Martinez-Maldonado, R. (2019). Human-Centred Learning Analytics. Journal of Learning Analytics , 6(2), 1–9. https://doi.org/10.18608/jla.2019.62.1 Carless, D. (2019). Feedback loops and the longer-term: Towards feedback spirals. Assessment & Evaluation in Higher Education, 44(5), 705–714. https://doi.org/10.1080/02602938.2018.1531108 Clarke, V., & Braun, V. (2014). Thematic Analysis. In: Encyclopedia of Quality of Life and Well-Being Research (pp. 6626–6628). Springer. Clark, T., Foster, L., Sloan, L., & Bryman, A. (2021b). Sampling in Qualitative Research. In: Bryman’s Social Research Methods (6th ed., pp. 1234–1283). Oxford. Clow, D. (2012). The learning analytics cycle: Closing the loop effectively. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge , 134–138. https://doi.org/10.1145/2330601.2330636 Dimitriadis, Y., Martínez-Maldonado, R., & Wiley, K. (2021). Human-Centered Design Principles for Actionable Learning Analytics. In T. Tsiatsos, S. Demetriadis, A. Mikropoulos, & V. Dagdilelis (Eds.), Research on E-Learning and ICT in Education (pp. 277–296). Springer International Publishing. https://doi.org/10.1007/978-3-030-64363-8_15 Crabtree, A., Rouncefield, M., & Tolmie, P. (2013). Doing design ethnography . Springer. Echeverria, V., Martinez-Maldonado, R., Buckingham Shum, S., Chiluiza, K., Granda, R., & Conati, C. (2018). Exploratory versus Explanatory Visual Learning Analytics: Driving Teachers’ Attention through Educational Data Storytelling. Journal of Learning Analytics , 5(3). https://doi.org/10.18608/jla.2018.53.6 Fernandez-Nieto, G. M., Martinez-Maldonado, R., Echeverria, V., Kitto, K., Gašević, D., & Buckingham Shum, S. (2024). Data Storytelling Editor: A Teacher-Centred Tool for Customising Learning Analytics Dashboard Narratives. Proceedings of the 14th Learning Analytics and Knowledge Conference, 678–689. https://doi.org/10.1145/3636555.3636930 Gašević, D., Kovanović, V., & Joksimović, S. (2017). Piecing the learning analytics puzzle: A consolidated model of a field of research and practice. Learning: Research and Practice , 3 (1), 63–78. https://doi.org/10.1080/23735082.2017.1286142 Giacomin, J. (2014). What Is Human Centred Design? The Design Journal , 17(4), 606–623. https://doi.org/10.2752/175630614X14056185480186 Harrington, S., Klein, S., Fratoni, A. C., Sellberg, C., & Lindwall, O. (2025). Meaningful Automated Feedback? Balancing Tensions in Participatory Learning Analytics Design. 18th International Conference on Computer-Supported Collaborative Learning, Helsinki, Finland. Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research , 77 (1), 81–112. https://doi.org/10.3102/003465430298487 Karahalil, M., Lützhöft, M., & Scanlan, J. (2023). Formative assessment in maritime simulator-based higher education. WMU Journal of Maritime Affairs , 22(2), 181-207. https://doi.org/10.1007/s13437-023-00313-6 Klerkx, J., Verbert, K., & Duval, E. (2017). Learning Analytics Dashboards. In C. Lang, G. Siemens, A. Wise, & D. Gasevic (Eds.), Handbook of Learning Analytics (pp. 143–150). Society for Learning Analytics Research (SoLAR). https://doi.org/10.18608/hla17.012 Le, L. H., Nguyen, H. D., Crane, M., & Mai, T. T. (2024). Multimedia learning analytics feedback in simulation-based training: A brief review. Proceedings of the 1st ACM Workshop on AI-Powered Q&A Systems for Multimedia , 25–30. https://doi.org/10.1145/3643479.3662053 Lim, L., Dawson, S., Joksimovic, S., & Gašević, D. (2019). Exploring students’ sensemaking of learning analytics dashboards: Does frame of reference make a difference? Proceedings of the 9th International Conference on Learning Analytics & Knowledge , 250–259. https://doi.org/10.1145/3303772.3303804 Martinez-Maldonado, R., Echeverria, V., Fernandez Nieto, G., & Buckingham Shum, S. (2020). From Data to Insights: A Layered Storytelling Approach for Multimodal Learning Analytics. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems , 1–15. https://doi.org/10.1145/3313831.3376148 Masiello, I., Mohseni, Z., Palma, F., Nordmark, S., Augustsson, H., & Rundquist, R. (2024). A Current Overview of the Use of Learning Analytics Dashboards. Education Sciences , 14(82), 1–15. https://doi.org/10.3390/educsci14010082 Ochoa, X. (2022). Multimodal Learning Analytics: Rationale, Process, Examples, and Direction. In: C. Lang, G. Siemens, & A. F. Wise (Eds.), The Handbook of Learning Analytics (2nd ed., pp. 54–65). SOLAR. https://doi.org/10.18608/hla22.006 Paulsen, L., & Lindsay, E. (2024). Learning analytics dashboards are increasingly becoming about learning and not just analytics—A systematic review. Education and Information Technologies , 29 (11), 14279–14308. https://doi.org/10.1007/s10639-023-12401-4 Roberts, L. D., Howell, J. A., & Seaman, K. (2017). Give Me a Customizable Dashboard: Personalized Learning Analytics Dashboards in Higher Education. Technology, Knowledge and Learning, 22(3), 317–333. https://doi.org/10.1007/s10758-017-9316-1 Ryan, L. (2016). The Visual Imperative: Creating a Visual Culture of Data Discovery (1 st edition). Elsevier Science & Technology. Schumacher, C., & Ifenthaler, D. (2018). Features students really expect from learning analytics. Computers in Human Behavior , 78, 397–407. https://doi.org/10.1016/j.chb.2017.06.030 Sellberg, C. & Sharma, A. (2024). Towards multimodal learning analytics in simulation-based collaborative learning: A design ethnography of maritime training. International Journal of Computer Supported Collaborative Learning, 20, 201–221. Sharma, K., & Giannakos, M. (2020). Multimodal data capabilities for learning: What can multimodal data tell us about learning? British Journal of Educational Technology , 51 (5), 1450–1484. https://doi.org/10.1111/bjet.12993 SOLAR. (2025). What is Learning Analytics? SOLAR - SOCIETY for LEARNING ANALYTICS RESEARCH. https://www.solaresearch.org/about/what-is-learning-analytics/ Spinuzzi, C. (2005). The Methodology of Participatory Design. Technical Communication , 52, 163–174. Verbert, K., Duval, E., Klerkx, J., Govaerts, S., & Santos, J. L. (2013). Learning Analytics Dashboard Applications. American Behavioral Scientist , 57 (10), 1500–1509. https://doi.org/10.1177/0002764213479363 Verbert, K., Duval, E., Santos, J. L., Van Assche, F., Parra, G., Klerkx, J., & Govaerts, S. (2013). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing , 18 , 1499–1514. https://doi.org/10.1007/s00779-013-0751-2 Vieira, C., Parsons, P., & Byrd, V. (2018). Visual learning analytics of educational data: A systematic literature review and research agenda. Computers & Education , 122, 119–135. https://doi.org/10.1016/j.compedu.2018.03.018 Villagrán, I., Hernández, R., Schuit, G., Neyem, A., Fuentes, J., Larrondo, L., Margozzini, E., Hurtado, M. T., Iriarte, Z., Miranda, C., Varas, J., & Hilliger, I. (2024). Enhancing Feedback Uptake and Self-Regulated Learning in Procedural Skills Training: Design and Evaluation of a Learning Analytics Dashboard. Journal of Learning Analytics , 11 (2), 138–156. https://doi.org/10.18608/jla.2024.8195 Wiley, K., Dimitriadis, Y., & Linn, M. (2024). A human-centred learning analytics approach for developing contextually scalable K‐12 teacher dashboards. British Journal of Educational Technology , 55(3), 845–885. https://doi.org/10.1111/bjet.13383 Wise, A. F., Knight, S., & Ochoa, X. (2021). What Makes Learning Analytics Research Matter. Journal of Learning Analytics , 8(3), 1–9. https://doi.org/10.18608/jla.2021.7647 Wong, B. T., & Li, K. C. (2020). A review of learning analytics intervention in higher education (2011–2018). Journal of Computers in Education , 7(1), 7–28. https://doi.org/10.1007/s40692-019-00143-7 Worsley, M., Anderson, K., Melo, N., & Jang, J. (2021). Designing Analytics for Collaboration Literacy and Student Empowerment. Journal of Learning Analytics , 8 (1), 30–48. https://doi.org/10.18608/jla.2021.7242 Yan, L., Echeverria, V., Jin, Y., Fernandez‐Nieto, G., Zhao, L., Li, X., ... & Martinez‐Maldonado, R. (2024). Evidence‐based multimodal learning analytics for feedback and reflection in collaborative learning. British Journal of Educational Technology , 55(5), 1900-1925. https://doi.org/10.1111/bjet.13498 Footnotes SMCP are a set of pre-defined English phrases used in maritime communication to ensure clear and unambiguous messaging, especially in situations where language barriers may exist. These phrases are part of the International Maritime Organization (IMO) standards. GMDSS is an internationally agreed-upon set of safety procedures, equipment, and communication protocols designed to enhance maritime safety and improve search and rescue operations for ships, boats, and aircraft in distress. COLREGs, or the International Regulations for Preventing Collisions at Sea, is a set of rules that govern the behaviour of vessels at sea to avoid collisions. Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7408850","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":502576531,"identity":"01abc061-a4d0-4a0d-984e-06a110180811","order_by":0,"name":"Sonja Klein","email":"","orcid":"","institution":"University of Gothenburg","correspondingAuthor":false,"prefix":"","firstName":"Sonja","middleName":"","lastName":"Klein","suffix":""},{"id":502576532,"identity":"43ee6ad5-30d1-4abb-82ed-1dc1c5969706","order_by":1,"name":"Charlott Sellberg","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABF0lEQVRIie2QPUvEQBCGx0abwLax0PyFOVIJ4v+w20XINQsKNik0BIRY2p7cr5BrrhxYuDQLtoIWCYL1iq2gsxc/Cpc7S5F9IJnlhYd3dwAikb9Iyl83HDeor3ls1UBrFQnoB5DySkJAK52fSiphZY2YXjw5Wb5WIAyRmu8fZ4ePvXFwrmpOgiUPi3wiLbekhSRli72b+wL5Yq2qOQkpmMocVOOVBEk1BkdT6ZUFKwmGlfHLoIhbx8objq5bNyichBX90QKan98QZr6O4IxXocMLu9OnIG2+3aT8BGWPEBN9QhYp90lwY5PxDFy5K4Qwffc8P8Dssp25sqx2roTpgjWfbH5dlZZ/852sJauXo/q1EIlEIv+ed/ynafl1u7M0AAAAAElFTkSuQmCC","orcid":"","institution":"University of Gothenburg","correspondingAuthor":true,"prefix":"","firstName":"Charlott","middleName":"","lastName":"Sellberg","suffix":""}],"badges":[],"createdAt":"2025-08-19 12:58:56","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-7408850/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7408850/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":89457816,"identity":"079c741e-5c5c-470f-b904-145d41b912dd","added_by":"auto","created_at":"2025-08-20 07:19:13","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":85384,"visible":true,"origin":"","legend":"\u003cp\u003eResearch Process\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-7408850/v1/7c211a6225751f235bfceac1.png"},{"id":89457819,"identity":"06cbe935-dc27-4034-8a0f-89a8608d96cc","added_by":"auto","created_at":"2025-08-20 07:19:13","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":2342069,"visible":true,"origin":"","legend":"\u003cp\u003eStudent-Cabin showing two radio simulators\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-7408850/v1/8a6e19c18c8b9efe50916441.png"},{"id":89457822,"identity":"fcb3f5f5-4b83-4197-afca-08b7bbdd1c82","added_by":"auto","created_at":"2025-08-20 07:19:13","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":228390,"visible":true,"origin":"","legend":"\u003cp\u003eBase Prototype\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-7408850/v1/69226572d381d78012f46c4c.png"},{"id":89458778,"identity":"da757d94-cecc-49c5-a040-161d11b2f46f","added_by":"auto","created_at":"2025-08-20 07:27:13","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":134847,"visible":true,"origin":"","legend":"\u003cp\u003eColor-Coded Graphs\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-7408850/v1/5f8aeb1273f3b3e8e0d0e620.png"},{"id":89459057,"identity":"7613f6b5-2303-4d18-ab83-0750c748bea6","added_by":"auto","created_at":"2025-08-20 07:35:13","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":30374,"visible":true,"origin":"","legend":"\u003cp\u003eColour Coded Audio Transcripts\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-7408850/v1/1c13366421decde7acf33b6a.png"},{"id":89457820,"identity":"1dd3c6cf-5f4c-47eb-9405-145b2bef6277","added_by":"auto","created_at":"2025-08-20 07:19:13","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":73970,"visible":true,"origin":"","legend":"\u003cp\u003eScaffolded Feedback for the category \"Filler Words\"\u003c/p\u003e","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-7408850/v1/7cfff6929a0614e9f4263b73.png"},{"id":89458777,"identity":"eaa15956-92e0-471e-8d25-ef7821f25e69","added_by":"auto","created_at":"2025-08-20 07:27:13","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":81333,"visible":true,"origin":"","legend":"\u003cp\u003eReflective Self-Assessment in the form of single-choice questions for the category \"Speed \u0026amp; Pauses\"\u003c/p\u003e","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-7408850/v1/7571a44f99583b46b13ce77e.png"},{"id":89459910,"identity":"819700eb-f19f-4d4a-84b3-f99e74bc147f","added_by":"auto","created_at":"2025-08-20 07:43:14","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3696378,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7408850/v1/ababb02a-af63-40ad-b89e-99d1d9edb7bd.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eClosing the Learning Analytics Loop through Explanatory Dashboard Design\u003c/p\u003e","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eIn recent years, \u003cem\u003eLearning Analytics\u003c/em\u003e (LA) has gained momentum across educational settings, spurring interest in how data-driven tools can meaningfully enhance student learning (Alfredo et al., 2024). A central strategy in this effort has been the use of \u003cem\u003eLearning Analytics Dashboards\u003c/em\u003e (LADs), which visualize learners\u0026rsquo; data to promote reflection, self-regulation, and awareness (Paulsen \u0026amp; Lindsay, 2024). While LADs hold significant promise, their pedagogical impact remains limited, particularly in complex, open-ended learning environments such as simulation-based training (Buckingham Shum et al., 2019). One recurring critique is that LADs often present decontextualized or overly generic data, which students may struggle to interpret or connect to their own learning processes (Masiello et al., 2024). This challenge is particularly pressing in simulation-based collaborative learning, where learning how to think and act as a professional is highly situated and embodied, shaped by the social and material contingencies of specific situations (Sellberg \u0026amp; Sharma, 2024). \u003cem\u003eMulti-Modal Learning Analytics\u003c/em\u003e (MMLA) has been identified to address this challenge by capturing and integrating diverse data modalities, such as speech, gesture, gaze, and analogue tool use, that reflect the embodied and interactional nature of professional learning. By moving beyond clickstream data to encompass the embodied and communicative dimensions of simulation-based activity, MMLA offers a foundation for designing dashboards that can represent learning in these environments and support students\u0026rsquo; reflective engagement (Yan et al., 2024).\u003c/p\u003e\n\u003cp\u003eThis paper addresses the challenge of supporting students\u0026rsquo; sensemaking of LADs in simulation-based maritime training. Drawing on a \u003cem\u003eHuman-Centred Learning Analytics\u003c/em\u003e (HCLA) perspective (Buckingham Shum et al. 2019), we argue that dashboards should not merely visualize performance metrics but be embedded within pedagogically meaningful structures. To this end, we build on the concept of \u003cem\u003eEducational Data Storytelling\u003c/em\u003e (EDS), which emphasizes the need to contextualize data through explanatory features aligned with the learning design (Echeverria et al. 2018). Rather than assuming students can independently make sense of complex dashboard data, our approach advocates for the co-design of LADs that reflect the temporal and pedagogical phases of simulation training, specifically, the sequence of briefing, scenario execution, and debriefing commonly used in maritime education. The study contributes to the growing body of HCLA research by examining how the feedback loop between performance and reflection can be more effectively closed. Specifically, we ask a) how student-facing explanatory dashboards can be designed to support sensemaking through contextualized data presentation, and b) what types of feedback and narrative elements foster reflective engagement in simulation-based learning environments.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe design case examined in this study is a simulation-based maritime communication course within a one-year maritime officer program at a Scandinavian university. The course focuses on the Global Maritime Distress and Safety System (GMDSS), where students are trained to use standardized phraseology and radio protocols in safety-critical scenarios. This setting offers a particularly productive context for exploring explanatory and student-facing learning analytics dashboards for two main reasons. First, the maritime domain\u0026rsquo;s reliance on highly structured communication protocols makes it amenable to automated analysis and feedback. Second, the pedagogical structure of the course, organized into briefing, scenario, and debriefing phases aligns well with narrative-based feedback models such as EDS, allowing dashboard elements to be integrated into existing pedagogical activities. Through participatory design methods (Bannon \u0026amp; Ehn, 2012), we explored how explanatory LAD features, such as feedback messages, visual scaffolds, and narrative cues, can support students in interpreting their own performance and engaging in self-reflection. By integrating EDS with a user-centred design approach, the study offers both conceptual and practical contributions. Conceptually, the study expands the use of narrative and feedback design within student-facing LADs in simulation-based professional training. Practically, it presents tangible, co-developed design features that illustrate the value of aligning learning analytics with pedagogical practice in simulation-based education.\u003c/p\u003e\n\u003cp\u003eThe remainder of the article is structured as follows. Section 2 reviews relevant literature on LA, MMLA, LADs, and design principles for explanatory, student-facing tools. Section 3 outlines the theoretical framing, combining HCLA and EDS. Section 4 describes the design case and methodology, including ethnographic observations, participatory design workshops, and prototype development. Section 5 presents the empirical findings, detailing instructors\u0026rsquo; feedback practices, co-designed feedback structures, and the resulting LAD prototype. Section 6 discusses the implications of the findings for learning analytics design in professional education, and Section 7 concludes the study.\u003c/p\u003e"},{"header":"2. Background","content":"\u003cp\u003eThe background section provides an overview of LA and MMLA (Section \u003cspan refid=\"Sec2\" class=\"InternalRef\"\u003e2.1\u003c/span\u003e), as well as LADs, focusing on the challenges involved in designing student-facing LADs that provide meaningful feedback and facilitating self-reflection amongst students (Section \u003cspan refid=\"Sec3\" class=\"InternalRef\"\u003e2.2\u003c/span\u003e). Lastly, existing design guidelines for LADs are outlined in Section \u003cspan refid=\"Sec4\" class=\"InternalRef\"\u003e2.3\u003c/span\u003e.\u003c/p\u003e\u003cdiv id=\"Sec2\" class=\"Section2\"\u003e\u003ch2\u003e2.1 Learning Analytics and Multi-Modal Learning Analytics\u003c/h2\u003e\u003cp\u003eLearning Analytics (LA) has emerged as a prominent research field aimed at improving learning and teaching through data-driven insights (Alfredo et al. \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). Defined in 2011 at the first International Conference on Learning Analytics and Knowledge (LAK), LA involves \u0026ldquo;\u003cem\u003ethe measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs\u003c/em\u003e\u0026rdquo; (SOLAR, \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e2025\u003c/span\u003e). As a field, LA emphasizes a holistic systems perspective and has predominantly focused on leveraging computational methods to explore and support social and educational processes (Baker \u0026amp; Siemens, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e2014\u003c/span\u003e). A central debate within LA concerns what constitutes meaningful data and how learning should be conceptualized. Early LA research relied heavily on easily accessible online activity data, often grounded in behaviourist assumptions (Verbert et al., \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e2013\u003c/span\u003e). However, this focus risks producing a \u0026ldquo;streetlight effect,\u0026rdquo; wherein attention is restricted to measurable activities rather than to contexts where LA could offer real pedagogical value (Ochoa, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Others, like Blikstein and Worsley (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2016\u003c/span\u003e), have highlighted the challenges of applying reductive data models in constructivist or socio-cultural learning contexts, which often feature open-ended and complex learning processes. To move beyond these limitations, Ochoa (\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e2022\u003c/span\u003e) calls for the integration of diverse data streams and context-specific behavioural markers, offering a pathway to more meaningful LA systems.\u003c/p\u003e\u003cp\u003eAddressing this need, Multi-Modal Learning Analytics (MMLA) has emerged as a subfield of LA, focused on expanding LA into collaborative and open-ended educational settings. As defined by Worsley et al. (2016), MMLA \u0026ldquo;\u003cem\u003eutilizes and triangulates among non-traditional as well as traditional forms of data in order to characterize or model student learning in complex learning environments\u003c/em\u003e\u0026rdquo; (p. 1346). First introduced by Blikstein (2013), MMLA responds to the limitations of earlier LA approaches by recognizing the inherently multimodal nature of learning, particularly within student-centred and open-ended pedagogies. In such settings, MMLA incorporates a variety of sensor technologies, including video, audio, bio signals, and motion capture, to record and analyse aspects of learning previously difficult to observe. These data are processed into meaningful modalities depending on the learning context, for example, through speech-to-text transcription or gesture recognition (Ochoa, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). This allows for a holistic analysis of learning, ultimately making formative feedback and assessment more relevant and actionable (Blikstein \u0026amp; Worsley, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2016\u003c/span\u003e). Moreover, MMLA has been positioned as a bridge between Learning Analytics and Learning Design (LD), ensuring that analytics align with pedagogical intentions (Sharma \u0026amp; Giannakos, \u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e2020\u003c/span\u003e). For instance, recent research in simulation-based education shows that when MMLA tools are meaningfully integrated into instructional practice, they can offer educators valuable, context-sensitive insights (Echeverria et al., \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2018\u003c/span\u003e).\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\u003ch2\u003e2.2. Closing the Loop through Learning Analytics Dashboard Design\u003c/h2\u003e\u003cp\u003eLearning Analytics Dashboards (LADs) are central interfaces through which insights from learning analytics (LA) are returned to students, educators, and other stakeholders. As visualization tools, LADs present traces of learning activity intending to foster awareness, reflection, sensemaking, and goal setting (Verbert et al., \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e2013\u003c/span\u003e). Ideally, LADs are designed not simply to display data, but to make that data pedagogically actionable. Paulsen and Lindsay (\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e2024\u003c/span\u003e) identify comparison, awareness, and monitoring as the most common design elements in LADs, often supplemented by features supporting goal-setting, recommendations, and self-assessment. Similarly, Schuhmacher and Ifenthaler (2018) highlight learners\u0026rsquo; preference for timely, actionable feedback, suggesting that LADs are most effective when they inform learning in the moment rather than after the activity.\u003c/p\u003e\u003cp\u003eLADs typically operate at three levels: descriptive (what happened), diagnostic (why it happened), and predictive (what might happen next or what to do). While descriptive LADs dominate current implementations, diagnostic and predictive versions remain underutilized (Paulsen \u0026amp; Lindsay, \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). In response, researchers have begun to develop explanatory LADs that offer interpretive guidance. Echeverria et al. (\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2018\u003c/span\u003e), for example, argue for contextualized, task-aligned dashboards, drawing an analogy to car dashboards that selectively present only what is needed for the current driving conditions. Similarly, Worsley et al. (\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e2021\u003c/span\u003e) co-designed a LAD with students for collaborative teamwork, embedding visual cues such as color-coded scripts and interaction timelines based on students\u0026rsquo; own criteria for success. These examples point toward situated and learner-centred approaches to LAD design.\u003c/p\u003e\u003cp\u003eConceptually, LADs are often framed within the broader logic of the LA \u0026ldquo;loop\u0026rdquo;, a cyclical process where data is collected from learner activity, analysed, and fed back to inform future learning. Clow (\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2012\u003c/span\u003e) introduced this loop as a model to emphasize that the value of LA lies not in data collection alone, but in how those data lead to meaningful changes in learning behaviour. The loop is only \u0026ldquo;closed\u0026rdquo; when the insights delivered through tools like LADs lead to pedagogically meaningful interventions, whether initiated by learners themselves or by educators. Wise et al. (\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e2021\u003c/span\u003e) reinforce this view, noting that LA must ultimately be evaluated in terms of its pedagogical impact. Unless insights are acted upon, LA systems remain inert, regardless of how sophisticated their analytics may be. While Clow (\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2012\u003c/span\u003e) stresses the importance of student understanding, he also argues that pedagogical value can still be achieved if faculty respond productively to the data. Wong and Li\u0026rsquo;s (\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e2020\u003c/span\u003e) review illustrates the variety of possible interventions, ranging from efforts to improve retention and academic performance to strategies for fostering engagement and deeper learning. Still, as Wise et al. (\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e2021\u003c/span\u003e) point out, the challenge of \u0026ldquo;closing the loop\u0026rdquo; remains a persistent blind spot in both research and practice.\u003c/p\u003e\u003cp\u003eThis study addresses that gap by focusing on how LADs are not only designed but used, particularly how their structure supports or constrains, learners\u0026rsquo; ability to make sense of data and act upon it. In this regard, the pedagogical value of LADs depends not only on their visual or technical sophistication but also on whether students can interpret and respond to the information they present. In the following section, we explore the conditions under which students make sense of LADs and the factors that support or hinder this process.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec4\" class=\"Section2\"\u003e\u003ch2\u003e2.3 Design Guidelines for Learning Analytics Dashboards\u003c/h2\u003e\u003cp\u003eThis section focuses on design strategies relevant to developing explanatory, student-facing Learning Analytics Dashboards (LADs), grounded in critical literature but emphasizing those that directly inform the design space explored here. Central to this synthesis is information visualization, a foundational approach in LAD development aimed at simplifying complex data through cognitive pattern discovery (Klerkx et al., \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). Although this approach might appear to conflict with the view of students as active agents within human-centred design, aligning visualization techniques with learning theory is essential to maximize pedagogical effectiveness (Vieira et al., \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e2018\u003c/span\u003e).\u003c/p\u003e\u003cdiv id=\"Sec5\" class=\"Section3\"\u003e\u003ch2\u003e2.3.1 Visual Structuring\u003c/h2\u003e\u003cp\u003eLADs typically leverage visual elements, colour, text formatting, layout, and representations like charts, maps, or flow diagrams to enhance interpretability (Le et al., \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e2024\u003c/span\u003e; Masiello et al., \u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). Adding contextual cues, such as inferential text or data labels, improves comprehension (Alhadad, \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2018\u003c/span\u003e; Wiley et al., \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). For simulation-based exercises, sequential visualizations like flowcharts help students reflect on decision-making processes within dynamic scenarios (Le et al., \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). While visualizations must be pedagogically driven (Masiello et al., \u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e2024\u003c/span\u003e), colour use is nuanced: it can direct attention but risks biasing interpretation if not carefully designed (Ahn et al., 2019; Roberts et al., \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). Text formatting and layout techniques like decluttering and chunking support focus and readability (Alhadad, \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2018\u003c/span\u003e; Wiley et al., \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). Interactive elements, filters, links, guided tours, enable deeper data exploration without overwhelming users, with educational experts valuing such hierarchical navigation in LADs (Bodily \u0026amp; Verbert, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2017\u003c/span\u003e).\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec6\" class=\"Section3\"\u003e\u003ch2\u003e2.3.2 Comparisons and Contextualization\u003c/h2\u003e\u003cp\u003eGrounding LAD data in the learning context enhances relevance, often through links to curricula, learning goals, or assessment items (Wiley et al., \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). Comparisons with personal or class performance indicators are common (Villagr\u0026aacute;n et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2024\u003c/span\u003e), though student responses vary: while some find peer comparisons motivating, others perceive them as discouraging (Roberts et al., \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). Recommendations derived from data mining offer actionable next steps but often lack transparency, impacting trust and uptake (Bodily \u0026amp; Verbert, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). When framed with a rationale, as in teacher- and student-facing LADs, such recommendations receive positive feedback (Wiley et al., \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e2024\u003c/span\u003e; Villagr\u0026aacute;n et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2024\u003c/span\u003e).\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec7\" class=\"Section3\"\u003e\u003ch2\u003e2.3.3 Interpreting and Narrating Data\u003c/h2\u003e\u003cp\u003eData storytelling structures LAD content narratively to foster interpretation and reflection, often following a narrative arc with critical moments and resolutions (Asgari \u0026amp; Hurtut, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). Data storytelling has shown promise in explanatory LADs for complex data, aiding reflection and interpretation in fields such as nursing education (Echeverria et al., 2023). Layered narrative designs support instructor-led debriefings and increase student accountability (Martinez-Maldonado et al., \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e2020\u003c/span\u003e). Summative textual evaluations combining descriptions of students actions and group feedback further assist understanding, with AI-generated summaries emerging as a useful tool (Bodily \u0026amp; Verbert, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2017\u003c/span\u003e).\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec8\" class=\"Section3\"\u003e\u003ch2\u003e2.3.4 Feedback and Reflection\u003c/h2\u003e\u003cp\u003eThough not always framed as an explicit design strategy, feedback literacy underpins students\u0026rsquo; sensemaking of LADs and highlights feedback\u0026rsquo;s reflective function (Carless, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e2019\u003c/span\u003e; Hattie \u0026amp; Timperley, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2007\u003c/span\u003e). Effective feedback addresses three key questions: \u003cem\u003eWhere am I going? How am I going? Where to next?\u003c/em\u003e (feed up, feed back, feed forward). Feedback must align with learning goals and be actionable, contextualized, and matched to the learner\u0026rsquo;s understanding and task (Hattie \u0026amp; Timperley, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2007\u003c/span\u003e). Reflection, a critical aspect of feedback uptake, remains underexplored in LAD design, though self-assessment features have been implemented with mixed student engagement (Villagr\u0026aacute;n et al., \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e2024\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eIn sum, the literature underscores the need for explanatory, learner-centred LAD designs that close the learning analytics loop through integrated visualization, contextualization, narrative, and feedback strategies. The key challenge lies in translating these theoretical strategies into tangible, pedagogically meaningful dashboard features that foster student reflection and sensemaking.\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e"},{"header":"3. Theoretical Framing","content":"\u003cp\u003eThis study draws on the combined perspectives of Human-Centered Learning Analytics (HCLA) and Educational Data Storytelling (EDS) to inform both the design and evaluation of student-facing dashboards. These frameworks are united in their emphasis on learner agency, pedagogical alignment, and the interpretability of data within authentic educational settings. \u003cem\u003eHuman-Centered Learning Analytics\u003c/em\u003e (HCLA) has been described as a counter-position to technically driven learning analytics systems, arguing that \u0026ldquo;it is more sensible to change the tools to suit their users, rather than changing the users to suit the tools\u0026rdquo; (Buckingham Shum et al., \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2019\u003c/span\u003e, p. 5). HCLA foregrounds learners as co-constructors of meaning, rather than passive recipients of feedback, and emphasizes that LA tools must be aligned with the pedagogical and contextual affordances of the learning environment (Dimitriadis et al., \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e2021\u003c/span\u003e). Key commitments of HCLA include the agency of stakeholders, integration with learning design, and grounding in educational theory. As emphasized, this approach views LA not just as a set of metrics, but as part of a socio-technical system designed in collaboration with users, particularly students and instructors (Alfredo et al., \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2024\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eTo complement this perspective, \u003cem\u003eEducational Data Storytelling\u003c/em\u003e (EDS) provides a design-oriented framework for embedding learning analytics in narrative structures. Developed as a response to the limitations of static visualizations, EDS scaffolds learners' interpretive process by aligning analytics with pedagogical intent and temporal learning trajectories (Echeverria et al., \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). Rather than offering exploratory dashboards that assume high levels of data literacy, EDS supports explanatory approaches that foreground selected data points and offer clear narrative cues to help students understand what happened, why it mattered, and how they can improve (Fern\u0026aacute;ndez-Nieto et al., 2024; Martinez-Maldonado et al., \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e2020\u003c/span\u003e). In line with this, EDS is positioned as a \u0026ldquo;middle ground\u0026rdquo; between personalized and generic dashboards, offering a balance between individualized insight and general contextual scaffolds (Echeverria et al., \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2018\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eBy combining HCLA and EDS, this study responds to the epistemological and practical challenges of designing learning analytics tools for professional learning environments. The joint application of these frameworks allows us to reconceptualize dashboards from monitoring tools into narrative spaces that support students\u0026rsquo; agency, reflection, and ongoing development. This reconceptualization is particularly relevant in simulation-based maritime training, where the reflective space created by debriefing plays a central role in learning. During debriefings, feedback typically unfolds as a narrative, drawing on domain-specific rules, prior experiences, and imagined future scenarios (Karahalil et al., \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e2023\u003c/span\u003e). In this way, debriefings align closely with the epistemic principles of EDS and offer a promising entry point for rethinking dashboard design. To explore this potential in context, we turn now to the setting in which our study was conducted.\u003c/p\u003e"},{"header":"4. The Design Process","content":"\u003cp\u003eThe participatory design process guiding this research follows Spinuzzi\u0026rsquo;s (\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e2005\u003c/span\u003e) three-stage model: \u003cem\u003eexploration\u003c/em\u003e, \u003cem\u003ediscovery\u003c/em\u003e, and \u003cem\u003eprototyping\u003c/em\u003e (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). The research method will be described in terms of these four interconnected stages. While presented as distinct for analytical purposes, the phases are inherently overlapping. In practice, prototyping occurs iteratively as part of a continuous effort to develop and refine educational designs (Cohen et al., 2011; Bannon \u0026amp; Ehn, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2012\u003c/span\u003e). Given that this study builds on prior design research, the process should be understood as part of an iterative design trajectory.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eBefore describing the method, an overview of the design case is provided.\u003c/p\u003e\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e\u003ch2\u003e4.1 The Design Case\u003c/h2\u003e\u003cp\u003eThe study examines a nine-week course in maritime communication within a one-year maritime officer program at a Scandinavian university, focusing on \u003cem\u003eGlobal Maritime Distress and Safety Systems\u003c/em\u003e (GMDSS). The course aimed to develop students\u0026rsquo; ability to communicate using maritime standard communication phrases (SMCP)[1]\u003ca class=\"FNLink\" href=\"#Fn1\" id=\"#FNLinkFn1\"\u003e\u003c/a\u003e and to operate GMDSS[2]\u003ca class=\"FNLink\" href=\"#Fn2\" id=\"#FNLinkFn2\"\u003e\u003c/a\u003e radio equipment. The course consists of theoretical lectures and simulation-based training. Weekly lectures introduced technical and operational content, including standard phraseology in Swedish and English, followed by mandatory simulator sessions.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eSimulator sessions took place in a dedicated radio simulator classroom featuring five student cabins and one instructor cabin, each equipped with Very High Frequency (VHF) and Digital Selective Calling (DSC) simulators (Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). These supported the simulation of distress and routine traffic communication across different systems. The pedagogical model followed a briefing-scenario-debriefing structure. Briefings introduced the scenario and equipment, while scenarios involved paired students simulating ship-to-shore communication, though the interaction was conducted individually. Each student enacted scripted procedures tailored to their ship\u0026rsquo;s identity, communicating with the instructor acting as a coastal radio station. The instructor occasionally paused the simulation to provide immediate feedback, and all sessions concluded with a debriefing that emphasized key learning points and professional relevance.\u003c/p\u003e\u003cp\u003eResearch ethics were carefully considered throughout the study, concerning privacy, consent, and potential risks to participants. As the simulator sessions reflected routine and non-graded learning activities, the risk of harm was assessed as minimal. All participants were fully informed about the aims and procedures of the study and gave their written consent before data collection. In the participatory design workshop, additional consent was obtained for the use of identifiable student video recordings, which were necessary to preserve the communicative nuances (e.g., gestures, posture, tone of voice) central to the feedback practices under investigation. Data were handled following local ethical guidelines, ensuring confidentiality, secure storage, and restricted access.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e\u003ch2\u003e4.2 Design Ethnography\u003c/h2\u003e\u003cp\u003eTo explore the situated dynamics of simulator-based learning, two video- and audio-recorded sessions were conducted as part of a design ethnographic study (Crabtree et al. \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e2013\u003c/span\u003e). The sessions took place over two days in the GDMSS simulator at a Scandinavian university, each session lasting approximately four hours. Although the same instructor facilitated both sessions, different student groups (8\u0026ndash;10 participants) attended. The video and audio recordings were supplemented by the collection of relevant artifacts, including curricula, learning objectives, and instructional materials, to contextualize the observed activities and feedback practices.\u003c/p\u003e\u003cp\u003eAccess was granted through the lead instructor, and session selection followed his recommendations. Aligned with Spinuzzi\u0026rsquo;s (\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e2005\u003c/span\u003e) emphasis on early-stage ethnographic exploration, this approach supported an open-ended examination of the enacted learning situation, with a focus on capturing feedback interactions and identifying candidate episodes for a future MMLA system. Field notes were used to support orientation within the audiovisual data corpus.\u003c/p\u003e\u003cp\u003eThematic analysis (Clarke \u0026amp; Braun, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2014\u003c/span\u003e) guided the post-observational phase. Via critical case sampling (Clark et al., 2021), eight incidents were selected for detailed analysis, covering briefings, in-action feedback, debriefings, and three instances of student activity, based on their relevance to instructional feedback. Transcription was performed using Whisper AI and manually reviewed for accuracy.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec13\" class=\"Section2\"\u003e\u003ch2\u003e4.3 Participatory Design Workshop\u003c/h2\u003e\u003cp\u003eTo triangulate observational findings and involve stakeholders in the design of a LAD with multimodal learning analytics, a participatory design workshop was conducted. The workshop aimed to co-design feedback rules that could later be implemented in the LAD (Echeverria, Martinez-Maldonado, Granda, et al., 2018). Four instructors in Maritime Communication from two European universities, recruited through purposive sampling (Clark et al., 2021) participated. During the 90-minute session, participants viewed three selected student video recordings and discussed the feedback they would normally provide. These sequences were shown unaltered, based on the relevance of body posture, gesture, and tone for interpreting feedback practices.\u003c/p\u003e\u003cp\u003eThe workshop discussion focused on identifying critical feedback moments and collaboratively exploring how such feedback is typically delivered and could be encoded as rules for automation. The session was transcribed in a denaturalized format (Oliver et al., 2005) using Whisper AI and manually validated. Thematic analysis (Clarke \u0026amp; Braun, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2014\u003c/span\u003e) was then conducted using NVivo, combining this material with data from the exploratory phase. Coding followed a deductive structure oriented around five W-questions, \u003cem\u003eWho, How, What, Why\u003c/em\u003e, and \u003cem\u003eWhen\u003c/em\u003e, resulting in six codes: learning context, learning content, learning intentions, feedback indicators, feedback message, and feedback deliverer. These were synthesized into four broader themes: learning intentions, indicators, timing, and feedback messages. Additionally, nine specific feedback targets emerged, including clarity of speech, number of transmissions, affective state, initial call structure, channel switching, and use of the Push-to-Talk (PTT) button. This analysis provided a granular understanding of how instructors structure feedback, which is essential for designing human-centred feedback in MMLA systems (Dimitriadis et al., \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e2021\u003c/span\u003e; Echeverria et al., \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2018\u003c/span\u003e).\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e\u003ch2\u003e4.4 Prototyping\u003c/h2\u003e\u003cp\u003eFollowing Lim et al. (2008), prototyping was seen as a method for evolving and exploring design ideas iteratively. The prototypes served as embedded artifacts to express and refine core pedagogical concepts, especially those related to EDS and feedback practices in simulator-based training. To narrow the scope while maintaining relevance, the prototyping focused on the theme of \u0026ldquo;clarity,\u0026rdquo; identified as both widely applicable in simulation contexts and particularly suited for enhancement through MMLA. The theme also aligned with student feedback expressing interest in improving the clarity of their speech. This focus helped balance generalizability with pedagogical grounding.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eThe base prototype was developed by digitizing earlier pen-and-paper designs using Figma, integrating prior feedback on usability aspects such as navigation and iconography (Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). A digital format was chosen to allow for interactive features, aligning with the complexity of EDS elements and facilitating meaningful stakeholder evaluation (Lim et al., \u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e2019\u003c/span\u003e). New design elements were developed through a 20-minute sketching session and further refined in an iterative process informed by earlier student and instructor input. The final interactive prototype included multimodal components such as graphs, audio feedback, reflective prompts, interactive questions, and color-coded transcript videos. The audio feedback was created using AI-generated voice recordings based on synthesized student speech scripts to protect participant privacy and enhance pedagogical richness by combining multiple learning indicators.\u003c/p\u003e\u003c/div\u003e"},{"header":"5. Results","content":"\u003cp\u003eThe findings show how instructors\u0026rsquo; feedback during simulation activities conveys specific learning intentions that can inform the design of feedback elements in LADs, and illustrate how these learning intentions, alongside the contextual feedback narratives in which they are embedded, can be translated into LAD features that support students\u0026rsquo; sensemaking. The results section outlines our empirical findings of feedback practices in GDMSS training (Section \u003cspan refid=\"Sec16\" class=\"InternalRef\"\u003e5.1\u003c/span\u003e), the instructors\u0026rsquo; feedback designs (Section \u003cspan refid=\"Sec17\" class=\"InternalRef\"\u003e5.2\u003c/span\u003e), and the prototype design (Section \u003cspan refid=\"Sec18\" class=\"InternalRef\"\u003e5.3\u003c/span\u003e).\u003c/p\u003e\u003cdiv id=\"Sec16\" class=\"Section2\"\u003e\u003ch2\u003e5.1 Feedback Practices in GDMSS Training\u003c/h2\u003e\u003cp\u003eDuring GMDSS simulation exercises, instructors frequently framed their feedback around students\u0026rsquo; use of maritime English, with an emphasis on clarity, linguistic efficiency, and adherence to standard communication protocols. For example, one recurring point of feedback concerned students\u0026rsquo; failure to say \u0026ldquo;\u003cem\u003eover\u003c/em\u003e\u0026rdquo; at the end of transmissions, which disrupted turn-taking and led to unnecessary repetitions. Instructors described this breakdown using metaphors, such as throwing a ball, to illustrate the correct turn-taking order. These kinds of feedback interactions reveal not only what students are expected to do but also the intentions that underpin these expectations.\u003c/p\u003e\u003cp\u003eThis section examines how such feedback practices can inform the development of feedback features in Learning Analytics Dashboards (LADs). By analyzing expected and observed student actions alongside instructor responses, we identify how contextualized, narrative-rich feedback can be translated into LAD elements that support students\u0026rsquo; reflective sensemaking. Where relevant, design suggestions from an instructor workshop are included, particularly concerning automated tracking and visualization of communication performance. Each feedback case is structured around the expected student action, the underlying learning intention, the observed action, and the instructor\u0026rsquo;s response. Where applicable, ideas from the instructor workshop regarding automation and visualization are also presented. Quotes from instructors (I1, I2, etc.) and the observed lecturer (L1) are included to illustrate specific insights while maintaining anonymity.\u003c/p\u003e\u003cp\u003eAcross the cases, instructors generally described their learning intentions as helping students practice maritime English, use standardized phrases, and become familiar with the GMDSS simulator environment. A central expectation was that students speak maritime English consistently throughout the exercises. One key focus was on the number of transmissions used in communication, which, according to European regulations, should match a targeted number. Deviations were often attributed to unclear turn endings, which instructors flagged as reducing communication efficiency. They expressed interest in LAD features that could visualize where transmission protocols, such as saying \u0026ldquo;\u003cem\u003eover\u003c/em\u003e\u0026rdquo;, were correctly or incorrectly followed.\u003c/p\u003e\u003cp\u003eA second learning intention centred on the clarity of speech, defined by tempo, pronunciation, and appropriate pausing between word groups, in line with IALA (2022) recommendations. Positive comments such as \u0026ldquo;\u003cem\u003egood tempo\u003c/em\u003e\u0026rdquo;, \u0026ldquo;\u003cem\u003eclear speech\u003c/em\u003e\u0026rdquo;, or \u0026ldquo;\u003cem\u003egood rhythm\u003c/em\u003e\u0026rdquo; were commonly used, though they tended to be brief and lacked narrative elaboration. In the workshop, instructors proposed including a clarity percentage or intelligibility score in the LAD, emphasizing the operational importance of being understood on the bridge. One instructor suggested that if speech was unintelligible, this alone should count as a feedback metric.\u003c/p\u003e\u003cp\u003eLinguistic efficiency emerged as a third key learning intention: students were expected to express messages using the fewest necessary words, avoiding fillers and repetitions. Despite this, instructors often observed filler words and digressions during simulation tasks. Rather than framing these as outright errors, they normalized them, \u0026ldquo;\u003cem\u003eIt\u0026rsquo;s normal. It\u0026rsquo;s there\u003c/em\u003e\u0026rdquo; (L1), and used these moments to introduce mitigating strategies, such as saying \u0026ldquo;standby\u0026rdquo; or releasing the Push-to-Talk button when thinking. During the workshop, instructors suggested including a feature in the LAD that compares the actual number of words used to the ideal phrasing based on the Standard Marine Communication Phrases: \u0026ldquo;\u003cem\u003eYou were trying to say this, it requires according to the SMCP code four words, and you used 28\u003c/em\u003e\u0026rdquo; (I1).\u003c/p\u003e\u003cp\u003eAcross all examples, instructor feedback, though mostly positive, frequently highlighted recurring issues through metaphors, case examples, or personal experience. These narrative elements not only made the feedback more accessible but also revealed instructional concerns that aligned closely with themes raised in the workshop. Together, these findings suggest that feedback in GMDSS training functions as a narrative practice grounded in situated learning intentions.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec17\" class=\"Section2\"\u003e\u003ch2\u003e5.2 Instructors\u0026rsquo; Feedback Designs\u003c/h2\u003e\u003cp\u003eIn the instructor workshop, participants reflected not only on the intentions behind their feedback but also on broader design considerations for integrating feedback into LADs. Overall, the instructors discussed the feasibility of visualizing student performance data, the potential for automating feedback messages, and the role of the LAD in supporting debriefings. A recurring theme in the conversation was the practical limitations of current feedback practices, particularly during more advanced simulations.\u003c/p\u003e\u003cp\u003eInstructors emphasized that certain feedback patterns were stable across sessions, making them suitable for automation. \u0026ldquo;\u003cem\u003eIf you generalize it a little bit\u0026hellip;\u0026rdquo;\u003c/em\u003e one instructor explained, \u0026ldquo;\u003cem\u003eyou always find the same good or bad\u0026hellip; or at least a lot of them. So those, you can create as a feedback bank\u003c/em\u003e\u0026rdquo; (I2). Rather than replacing instructor judgment, such a feedback bank could handle basic, recurrent issues, allowing instructors to focus their attention on more complex, situation-specific matters. This point was reinforced in discussions about workload. \u0026ldquo;\u003cem\u003eI don\u0026rsquo;t want to go into all the independent cabins and like, \u0026lsquo;okay, so look\u0026hellip;\u0026rsquo;, because there\u0026rsquo;s no time for that\u003c/em\u003e\u0026rdquo; (I1), one instructor noted, highlighting time constraints during simulations as a barrier to individual feedback.\u003c/p\u003e\u003cp\u003eThis challenge was especially relevant in relation to communication performance, which instructors described as frequently overlooked during assessments. As one instructor put it, \u0026ldquo;\u003cem\u003eAs long as you have some kind of communication that is understood, you don\u0026rsquo;t comment on it so much, because you will be focusing on COLREGs\u003c/em\u003e[3]\u003ca class=\"FNLink\" href=\"#Fn3\" id=\"#FNLinkFn3\"\u003e\u003c/a\u003e, \u003cem\u003eor whatever it is, right?\u003c/em\u003e\u0026rdquo; (I1). Here, the potential value of the LAD became particularly evident: instructors envisioned using it as a tool to foreground aspects, like communication clarity, that otherwise risk being neglected. They emphasized its value not as a stand-alone assessment tool, but as a prompt for post-exercise reflection. The LAD, they agreed, would be most useful during debriefings to support structured, data-driven discussions with students.\u003c/p\u003e\u003cp\u003eWhen discussing how feedback data might be presented in the LAD, instructors expressed mixed feelings about the idea of automated interpretation. Some worried about demotivating students: \u0026ldquo;\u003cem\u003eWhat if they just get thumbs down all the time?\u0026rdquo;\u003c/em\u003e (I1). Rather than labeling performance as \u0026ldquo;\u003cem\u003egood\u003c/em\u003e\u0026rdquo; or \u0026ldquo;\u003cem\u003ebad\u003c/em\u003e\u0026rdquo;, instructors advocated for neutral visualizations, such as ratios, percentages, or other forms of quantified data, that could serve as starting points for conversation. Final interpretations, they argued, should be left to instructors and negotiated with students.\u003c/p\u003e\u003cp\u003eStill, the instructors recognized the pedagogical value of certain interpretive features, especially if linked to students\u0026rsquo; own actions. One instructor was particularly enthusiastic about the potential of integrating audio playback with automatic analysis of speech: \u0026ldquo;\u003cem\u003eIf you can replay and listen to yourself\u0026hellip; that is a pedagogical masterpiece actually! And especially if the computer then [says]: \u0026lsquo;unnecessary,\u0026rsquo; \u0026lsquo;not according to SMCP\u0026rsquo;\u003c/em\u003e\u0026rdquo; (I1). Such a feature, they suggested, could make regulatory alignment visible and actionable, combining narrative and data to support learning, a central principle in EDS. During the workshop, instructors also offered concrete ideas for how feedback could be visualized. One proposed using line graphs to show changes over time. Others suggested layering information, using icons or short evaluative phrases, to make complex feedback easier to interpret at a glance.\u003c/p\u003e\u003cp\u003eTaken together, the workshop discussions show that instructors view the LAD primarily as a reflective tool to support feedback conversations, rather than a mechanism for judgment or grading. They were optimistic about the possibility of automating context-sensitive feedback messages, particularly those grounded in repeated patterns of student action. At the same time, they expressed caution about overly directive or interpretive LAD features, emphasizing the need for transparency, flexibility, and pedagogical alignment with their instructional goals.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec18\" class=\"Section2\"\u003e\u003ch2\u003e5.3 Prototype Design\u003c/h2\u003e\u003cp\u003eThe LAD prototype was developed to visualize feedback from simulator-based maritime communication training, with an initial design focus on clarity of speech. The base interface provides a session overview, including student role, course ID, date, and curriculum-based exercise title. Each session includes an audio playback of the communication activity and an overview of feedback categories, \u0026ldquo;\u003cem\u003eClarity\u003c/em\u003e\u0026rdquo;, \u0026ldquo;\u003cem\u003eInformation Exchange\u003c/em\u003e\u0026rdquo;, and \u0026ldquo;\u003cem\u003ePTT Checks\u003c/em\u003e\u0026rdquo;, displayed through interactive summary metrics and expandable trend graphs (see Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eIn its first iteration, only the \u0026ldquo;\u003cem\u003eClarity\u003c/em\u003e\u0026rdquo; category was elaborated in detail. This category was subdivided into \u0026ldquo;\u003cem\u003eSpeed \u0026amp; Pauses\u003c/em\u003e\u0026rdquo;, \u0026ldquo;\u003cem\u003eFiller Words\u003c/em\u003e\u0026rdquo;, and \u0026ldquo;\u003cem\u003eUnnecessary Repetitions\u003c/em\u003e\u0026rdquo;. Each sub-category is associated with a development graph, a five-session moving average, and trend indicators. Feedback is presented using an \u0026ldquo;\u003cem\u003eif-indicator-then-message\u003c/em\u003e\u0026rdquo; structure: when specific linguistic markers are detected (e.g., filler word use or speech rate deviation), feedback messages are generated to prompt learning insights. Feedback cases were categorized under the broader learning objective of promoting speech aligned with SMCP. For example, speech rated as too fast or lacking pauses is flagged under \u0026ldquo;\u003cem\u003eSpeed \u0026amp; Pauses\u003c/em\u003e\u0026rdquo;, while redundant phrasing or filler terms trigger feedback in their respective sub-categories. Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e illustrates how specific communicative features were mapped onto LAD categories, alongside the types of feedback responses they triggered (e.g., positive reinforcement, self-reflective prompts, SMCP references, or regulatory comparisons).\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eMapping of Category Clarity with the overall learning intention: \u0026ldquo;The student speaks clearly according to Standard Maritime Communications and Phrases Regulations.\u0026rdquo;\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"4\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u003cp\u003eFeedback Case\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eAnalysed Feedback Indicators\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003eAnalysed Feedback Messages\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c4\"\u003e\u003cp\u003eLAD Category\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eSpeaking clearly\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eStudent takes no breaks\u003c/p\u003e\u003cp\u003eStudent speaks faster or slower than 120 words per minute\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003ePositive feedback (e.g. \u0026rdquo;good speed\u0026rdquo;)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eSpeed \u0026amp; Pauses\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e\u003cp\u003eUsing minimum amount of words necessary\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eUse of filler words (e.g. \u0026ldquo;ehm\u0026rdquo;)\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\" morerows=\"1\" rowspan=\"2\"\u003e\u003cp\u003eJustification of normalcy\u003c/p\u003e\u003cp\u003eSelf-reflecting question\u003c/p\u003e\u003cp\u003eProposing coping strategy\u003c/p\u003e\u003cp\u003eReferral to SMCP regulations\u003c/p\u003e\u003cp\u003eComparison necessary words/words used\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eFiller Words\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eRepeating words unnecessarily\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eUnnecessary Repetitions\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\u003cdiv id=\"Sec19\" class=\"Section3\"\u003e\u003ch2\u003e5.1.1 Visual Feedback Design\u003c/h2\u003e\u003cp\u003eTo support longitudinal reflection on performance, interactive line and bar graphs were embedded in the dashboard. These visualizations track individual progress across multiple sessions, with session-specific data accessible via clickable nodes. In response to previous iterations of the prototype, a color-coded background was added in a second design iteration to support intuitive interpretation (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e). A traffic-light colour scheme (green/yellow/red) was used to denote performance zones. For instance, a rate near 120 words per minute was defined as optimal for clarity (green), while a low frequency of fillers indicated effective communication. Although instructors expressed reservations about over-reliance on visual coding, the colour scheme was retained due to its interpretive affordances for students unfamiliar with numerical metrics.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec20\" class=\"Section3\"\u003e\u003ch2\u003e5.1.2 Audio Transcript Playbacks\u003c/h2\u003e\u003cp\u003eDrawing on insights from earlier participatory design sessions with maritime students, where participants emphasized the importance of accessing and reviewing their simulator data (Harrington et al. \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e2025\u003c/span\u003e), the prototype integrates audio transcript playbacks that align line-by-line with students\u0026rsquo; audio-recorded communication. These playbacks highlight speech in real time, with color-coded transcripts indicating whether utterances matched or violated expected standards. Within \u0026ldquo;\u003cem\u003eFiller Words\u003c/em\u003e\u0026rdquo; and \u0026ldquo;\u003cem\u003eUnnecessary Repetitions\u003c/em\u003e\u0026rdquo;, system-detected infractions are marked in red, while compliant speech appears in green (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e).\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eTo complement the students\u0026rsquo; transcripts, model transcripts of correct performance were included. These are presented without colour markings but are accompanied by interpretive feedback to illustrate what students should have said. This dual representation, student output and a regulatory-conforming exemplar, enables comparative reflection and supports corrective learning.\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec21\" class=\"Section3\"\u003e\u003ch2\u003e5.1.3 Reflective Feedback and Scaffolded Self-Assessment\u003c/h2\u003e\u003cp\u003eInfo banners provide a textual summary of key data points, including what the student attempted, applicable SMCP regulations, and comparative evaluations. These banners were designed to bridge data representation and pedagogical interpretation, particularly for learners less comfortable navigating charts. They also serve as entry points to more detailed feedback layers.\u003c/p\u003e\u003cp\u003eTo mimic instructor-facilitated reflection during post-simulation debriefings, the prototype incorporates self-assessment features. These include open-ended prompts and multiple-choice questions embedded within specific categories. For example, in Filler Words, students are first shown an info banner summarizing observed speech behaviour and its misalignment with regulations. Upon expanding this section, they are prompted to reflect on possible coping strategies, with the correct answer withheld until the \u0026ldquo;show answer\u0026rdquo; option is clicked. This deliberate delay is designed to scaffold metacognitive engagement.\u003c/p\u003e\u003cp\u003eIn Speed \u0026amp; Pauses, incidents such as unusually long silences (e.g., a 22-second pause) are flagged, with students asked to select the most plausible explanation from three options. Feedback is delivered via visual outlines (red/green) and is followed by an interpretive explanation, often including metaphors or contextualized recommendations for improvement. These feedback layers collectively aim to foster student agency in interpreting data, integrating instructor-authored feedback cases into an interactive, student-centred learning analytics interface (Harrington et al. \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e2025\u003c/span\u003e).\u003c/p\u003e\u003c/div\u003e\u003c/div\u003e"},{"header":"6. Discussion","content":"\u003cp\u003eThis study set out to explore how reflective feedback features can be designed for LADs in simulator-based training, building on the principles of HCLA and EDS. The aim was to address a noted gap in LA and MMLA research: the need to support students not just with data points, but with pedagogically meaningful feedback that can foster sense-making and self-assessment (Echeverr\u0026iacute;a et al., 2018). Through a prototype focused on maritime radio communication, the study investigated how LADs might visualize and contextualize feedback in a way that resonates with both the structure of simulator-based learning and the social dynamics of feedback practices.\u003c/p\u003e\u003cp\u003eThe findings suggest that feedback in simulation contexts is highly narrative, situated, and oriented toward regulation-based communicative competence. Instructors\u0026rsquo; use of metaphors, case-based examples, and regulatory framings provides a natural bridge to EDS. In particular, the notion of \u0026ldquo;\u003cem\u003ethrowing the ball\u003c/em\u003e\u0026rdquo; to describe conversational turn-taking illustrates how instructors already embed narrative logic into their feedback to support students\u0026rsquo; understanding. This observation supports the use of LADs not merely as repositories of performance data but as tools that can extend such narratives through carefully designed visualizations and interactions. The incorporation of features like color-coded transcripts, audio playback, and reflective questions demonstrates how LADs can align with and enhance these narrative practices.\u003c/p\u003e\u003cp\u003eThe findings also reveal how instructors\u0026rsquo; existing feedback practices include pedagogical narratives that can be directly leveraged in LAD design. Rather than relying on generic indicators, instructors use situated metaphors, regulatory references, and normalizing framings to help students interpret their communication performance. These narratives, often repeated across student interactions, form a repertoire for context-aware automation, as instructors themselves noted. Their reflections pointed to a dual role for LADs: as tools for individualized reflection and as prompts for instructor-led discussion during debriefings. Importantly, instructors saw value in automating foundational feedback while reserving interpretive authority for more complex or ambiguous cases. This aligns with the broader ambition of explanatory dashboards: not to replace human feedback but to provide meaningful scaffolds (Echeverria et al., \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). Still, tensions remain around how much interpretation should be embedded in LADs. While instructors welcomed features like audio playback and SMCP-based benchmarking, the instructors expressed caution toward prescriptive judgments (e.g., thumbs-down icons), advocating instead for neutral data displays that support shared interpretation. These insights underscore the value of designing LADs that support professional judgment while expanding opportunities for learner reflection, an approach consistent with both HCLA and EDS frameworks.\u003c/p\u003e\u003cp\u003eThese findings also speak to a central tension in the learning analytics literature: the balance between interpretability and prescription (Buckingham Shum et al., 2022; Ifenthaler \u0026amp; Yau, 2020). While the instructors valued LADs that could automate aspects of feedback delivery, they resisted systems that might fix meanings or evaluations without space for negotiation. This reflects ongoing critiques of dashboards that impose normative models of learning without attending to the contextual, relational, and interpretive dimensions of educational practice. The workshop participants\u0026rsquo; interest in integrating narrative metaphors, student audio, and regulatory references illustrates how EDS can be operationalized in practice, not to deliver definitive judgments, but to scaffold students\u0026rsquo; and instructors\u0026rsquo; shared interpretive work. In this sense, the LAD is not just a feedback tool but a boundary object that mediates between data traces, professional teaching knowledge, and student reflection. Designing for such a role requires not only technical functionality but also humility towards what it means to learn in complex safety-critical domains: the willingness to foreground uncertainty, enable dialogue, and support human sense-making over algorithmic closure (Buckingham Shum et al., 2022; Mart\u0026iacute;nez-Maldonado, 2020).\u003c/p\u003e\u003cp\u003eTaken together, the study contributes to a growing body of literature calling for a shift from performance-focused dashboards toward reflective and dialogic tools that extend existing pedagogical practices. Overall, the study demonstrates how LADs can be meaningful not by delivering answers, but by prompting the kinds of questions instructors typically ask, and by giving students the tools to ask them of themselves.\u003c/p\u003e"},{"header":"7. Conclusion","content":"\u003cp\u003eThis study contributes to the field HCLA by demonstrating how carefully designed dashboards can function as meaningful reflective tools in simulation-based professional education. Through the integration of narrative feedback and contextualized data presentation, the prototype reimagines feedback in learning analytics not as abstract metrics, but as part of a coherent narrative structure that supports learners\u0026rsquo; sensemaking. The findings underscore the value of aligning feedback with the pedagogical practices of simulation training, enabling retrospective reflection grounded in concrete learning scenarios. In this way, the dashboard serves as a reflective space that promotes deeper engagement with one\u0026rsquo;s practice and supports ongoing professional development.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eAlhadad, S. S. J. (2018). Visualizing Data to Support Judgement, Inference, and Decision Making in Learning Analytics: Insights from Cognitive Psychology and Visualization Science. \u003cem\u003eJournal of Learning Analytics\u003c/em\u003e, 5(2), 60\u0026ndash;85. https://doi.org/10.18608/jla.2018.52.5\u003c/li\u003e\n\u003cli\u003eAlfredo, R., Echeverria, V., Jin, Y., Yan, L., Swiecki, Z., Ga\u0026scaron;ević, D., \u0026amp; Martinez-Maldonado, R. (2024). Human-centred learning analytics and AI in education: A systematic literature review. \u003cem\u003eComputers and Education: Artificial Intelligence\u003c/em\u003e, \u003cem\u003e6\u003c/em\u003e, 1\u0026ndash;23. https://doi.org/10.1016/j.caeai.2024.100215\u003c/li\u003e\n\u003cli\u003eAsgari, M., \u0026amp; Hurtut, T. (2024). A Design Language for Prototyping and Storyboarding Data-Driven Stories. \u003cem\u003eApplied Sciences\u003c/em\u003e, 14(4), 1387. https://doi.org/10.3390/app14041387\u003c/li\u003e\n\u003cli\u003eBaker, R., \u0026amp; Siemens, G. (2014). Educational Data Mining and Learning Analytics. In R. K. Sawyer (Ed.), \u003cem\u003eThe Cambridge Handbook of the Learning Sciences\u003c/em\u003e (2nd ed., pp. 253\u0026ndash;272). Cambridge University Press. https://doi.org/10.1017/CBO9781139519526.016\u003c/li\u003e\n\u003cli\u003eBannon, L. J., \u0026amp; Ehn, P. (2012). Design Matters in Participatory Design. In: \u003cem\u003eRoutledge International Handbook of Participatory Design\u003c/em\u003e (1st ed., pp. 37\u0026ndash;63). Routledge. https://doi.org/10.4324/9780203108543\u003c/li\u003e\n\u003cli\u003eBlikstein, P., \u0026amp; Worsley, M. (2016). Multimodal Learning Analytics and Education Data Mining: Using computational technologies to measure complex learning tasks. \u003cem\u003eJournal of Learning Analytics\u003c/em\u003e, 3(2), 220\u0026ndash;238. https://doi.org/10.18608/jla.2016.32.11\u003c/li\u003e\n\u003cli\u003eBodily, R., \u0026amp; Verbert, K. (2017). Trends and issues in student-facing learning analytics reporting systems research. \u003cem\u003eProceedings of the Seventh International Learning Analytics \u0026amp; Knowledge Conference\u003c/em\u003e, 309\u0026ndash;318. https://doi.org/10.1145/3027385.3027403\u003c/li\u003e\n\u003cli\u003eBuckingham Shum, S., Ferguson, R., \u0026amp; Martinez-Maldonado, R. (2019). Human-Centred Learning Analytics. \u003cem\u003eJournal of Learning Analytics\u003c/em\u003e, 6(2), 1\u0026ndash;9. https://doi.org/10.18608/jla.2019.62.1\u003c/li\u003e\n\u003cli\u003eCarless, D. (2019). Feedback loops and the longer-term: Towards feedback spirals. \u003cem\u003eAssessment \u0026amp; Evaluation in Higher Education,\u003c/em\u003e 44(5), 705\u0026ndash;714. https://doi.org/10.1080/02602938.2018.1531108\u003c/li\u003e\n\u003cli\u003eClarke, V., \u0026amp; Braun, V. (2014). Thematic Analysis. In: \u003cem\u003eEncyclopedia of Quality of Life and Well-Being Research\u003c/em\u003e (pp. 6626\u0026ndash;6628). Springer.\u003c/li\u003e\n\u003cli\u003eClark, T., Foster, L., Sloan, L., \u0026amp; Bryman, A. (2021b). Sampling in Qualitative Research. In: \u003cem\u003eBryman\u0026rsquo;s Social Research Methods\u003c/em\u003e (6th ed., pp. 1234\u0026ndash;1283). Oxford.\u003c/li\u003e\n\u003cli\u003eClow, D. (2012). The learning analytics cycle: Closing the loop effectively. \u003cem\u003eProceedings of the 2nd\u003c/em\u003e \u003cem\u003eInternational Conference on Learning Analytics and Knowledge\u003c/em\u003e, 134\u0026ndash;138. https://doi.org/10.1145/2330601.2330636\u003c/li\u003e\n\u003cli\u003eDimitriadis, Y., Mart\u0026iacute;nez-Maldonado, R., \u0026amp; Wiley, K. (2021). Human-Centered Design Principles for Actionable Learning Analytics. In T. Tsiatsos, S. Demetriadis, A. Mikropoulos, \u0026amp; V. Dagdilelis (Eds.), \u003cem\u003eResearch on E-Learning and ICT in Education\u003c/em\u003e (pp. 277\u0026ndash;296). Springer International Publishing. https://doi.org/10.1007/978-3-030-64363-8_15\u003c/li\u003e\n\u003cli\u003eCrabtree, A., Rouncefield, M., \u0026amp; Tolmie, P. (2013). \u003cem\u003eDoing design ethnography\u003c/em\u003e. Springer. \u003c/li\u003e\n\u003cli\u003eEcheverria, V., Martinez-Maldonado, R., Buckingham Shum, S., Chiluiza, K., Granda, R., \u0026amp; Conati, C. (2018). Exploratory versus Explanatory Visual Learning Analytics: Driving Teachers\u0026rsquo; Attention through Educational Data Storytelling. \u003cem\u003eJournal of Learning Analytics\u003c/em\u003e, 5(3). https://doi.org/10.18608/jla.2018.53.6\u003c/li\u003e\n\u003cli\u003eFernandez-Nieto, G. M., Martinez-Maldonado, R., Echeverria, V., Kitto, K., Ga\u0026scaron;ević, D., \u0026amp; Buckingham Shum, S. (2024). Data Storytelling Editor: A Teacher-Centred Tool for Customising Learning Analytics Dashboard Narratives. \u003cem\u003eProceedings of the 14th Learning Analytics and Knowledge Conference,\u003c/em\u003e 678\u0026ndash;689. https://doi.org/10.1145/3636555.3636930\u003c/li\u003e\n\u003cli\u003eGa\u0026scaron;ević, D., Kovanović, V., \u0026amp; Joksimović, S. (2017). Piecing the learning analytics puzzle: A consolidated model of a field of research and practice. \u003cem\u003eLearning: Research and Practice\u003c/em\u003e, \u003cem\u003e3\u003c/em\u003e(1), 63\u0026ndash;78. https://doi.org/10.1080/23735082.2017.1286142\u003c/li\u003e\n\u003cli\u003eGiacomin, J. (2014). What Is Human Centred Design? \u003cem\u003eThe Design Journal\u003c/em\u003e, 17(4), 606\u0026ndash;623. https://doi.org/10.2752/175630614X14056185480186\u003c/li\u003e\n\u003cli\u003eHarrington, S., Klein, S., Fratoni, A. C., Sellberg, C., \u0026amp; Lindwall, O. (2025). Meaningful Automated Feedback? Balancing Tensions in Participatory Learning Analytics Design. \u003cem\u003e18th International Conference on Computer-Supported Collaborative Learning,\u003c/em\u003e Helsinki, Finland.\u003c/li\u003e\n\u003cli\u003eHattie, J., \u0026amp; Timperley, H. (2007). The Power of Feedback. \u003cem\u003eReview of Educational Research\u003c/em\u003e, \u003cem\u003e77\u003c/em\u003e(1), 81\u0026ndash;112. https://doi.org/10.3102/003465430298487\u003c/li\u003e\n\u003cli\u003eKarahalil, M., L\u0026uuml;tzh\u0026ouml;ft, M., \u0026amp; Scanlan, J. (2023). Formative assessment in maritime simulator-based higher education. \u003cem\u003eWMU Journal of Maritime Affairs\u003c/em\u003e, 22(2), 181-207. https://doi.org/10.1007/s13437-023-00313-6\u003c/li\u003e\n\u003cli\u003eKlerkx, J., Verbert, K., \u0026amp; Duval, E. (2017). Learning Analytics Dashboards. In C. Lang, G. Siemens, A. Wise, \u0026amp; D. Gasevic (Eds.), \u003cem\u003eHandbook of Learning Analytics\u003c/em\u003e (pp. 143\u0026ndash;150). Society for Learning Analytics Research (SoLAR). https://doi.org/10.18608/hla17.012\u003c/li\u003e\n\u003cli\u003eLe, L. H., Nguyen, H. D., Crane, M., \u0026amp; Mai, T. T. (2024). Multimedia learning analytics feedback in simulation-based training: A brief review. \u003cem\u003eProceedings of the 1st ACM Workshop on AI-Powered\u003c/em\u003e \u003cem\u003eQ\u0026amp;A Systems for Multimedia\u003c/em\u003e, 25\u0026ndash;30. https://doi.org/10.1145/3643479.3662053\u003c/li\u003e\n\u003cli\u003eLim, L., Dawson, S., Joksimovic, S., \u0026amp; Ga\u0026scaron;ević, D. (2019). Exploring students\u0026rsquo; sensemaking of learning analytics dashboards: Does frame of reference make a difference? \u003cem\u003eProceedings of the 9th\u003c/em\u003e \u003cem\u003eInternational Conference on Learning Analytics \u0026amp; Knowledge\u003c/em\u003e, 250\u0026ndash;259. https://doi.org/10.1145/3303772.3303804\u003c/li\u003e\n\u003cli\u003eMartinez-Maldonado, R., Echeverria, V., Fernandez Nieto, G., \u0026amp; Buckingham Shum, S. (2020). From Data to Insights: A Layered Storytelling Approach for Multimodal Learning Analytics. \u003cem\u003eProceedings of the 2020 CHI Conference on Human Factors in Computing Systems\u003c/em\u003e, 1\u0026ndash;15. https://doi.org/10.1145/3313831.3376148\u003c/li\u003e\n\u003cli\u003eMasiello, I., Mohseni, Z., Palma, F., Nordmark, S., Augustsson, H., \u0026amp; Rundquist, R. (2024). A Current Overview of the Use of Learning Analytics Dashboards. \u003cem\u003eEducation Sciences\u003c/em\u003e, 14(82), 1\u0026ndash;15. https://doi.org/10.3390/educsci14010082\u003c/li\u003e\n\u003cli\u003eOchoa, X. (2022). Multimodal Learning Analytics: Rationale, Process, Examples, and Direction. In: C. Lang, G. Siemens, \u0026amp; A. F. Wise (Eds.), \u003cem\u003eThe Handbook of Learning Analytics\u003c/em\u003e (2nd ed., pp. 54\u0026ndash;65). SOLAR. https://doi.org/10.18608/hla22.006\u003c/li\u003e\n\u003cli\u003ePaulsen, L., \u0026amp; Lindsay, E. (2024). Learning analytics dashboards are increasingly becoming about learning and not just analytics\u0026mdash;A systematic review. \u003cem\u003eEducation and Information Technologies\u003c/em\u003e, \u003cem\u003e29\u003c/em\u003e(11), 14279\u0026ndash;14308. https://doi.org/10.1007/s10639-023-12401-4\u003c/li\u003e\n\u003cli\u003eRoberts, L. D., Howell, J. A., \u0026amp; Seaman, K. (2017). Give Me a Customizable Dashboard: Personalized Learning Analytics Dashboards in Higher Education. \u003cem\u003eTechnology, Knowledge and Learning,\u003c/em\u003e 22(3), 317\u0026ndash;333. https://doi.org/10.1007/s10758-017-9316-1\u003c/li\u003e\n\u003cli\u003eRyan, L. (2016). \u003cem\u003eThe Visual Imperative: Creating a Visual Culture of Data Discovery\u003c/em\u003e (1\u003csup\u003est\u003c/sup\u003e edition). Elsevier Science \u0026amp; Technology.\u003c/li\u003e\n\u003cli\u003eSchumacher, C., \u0026amp; Ifenthaler, D. (2018). Features students really expect from learning analytics. \u003cem\u003eComputers in Human Behavior\u003c/em\u003e, 78, 397\u0026ndash;407. https://doi.org/10.1016/j.chb.2017.06.030\u003c/li\u003e\n\u003cli\u003eSellberg, C. \u0026amp; Sharma, A. (2024). Towards multimodal learning analytics in simulation-based collaborative learning: A design ethnography of maritime training. \u003cem\u003eInternational Journal of Computer Supported Collaborative Learning,\u003c/em\u003e 20, 201\u0026ndash;221.\u003c/li\u003e\n\u003cli\u003eSharma, K., \u0026amp; Giannakos, M. (2020). Multimodal data capabilities for learning: What can multimodal data tell us about learning? \u003cem\u003eBritish Journal of Educational Technology\u003c/em\u003e, \u003cem\u003e51\u003c/em\u003e(5), 1450\u0026ndash;1484. https://doi.org/10.1111/bjet.12993\u003c/li\u003e\n\u003cli\u003eSOLAR. (2025). \u003cem\u003eWhat is Learning Analytics?\u003c/em\u003e SOLAR - SOCIETY for LEARNING ANALYTICS RESEARCH. https://www.solaresearch.org/about/what-is-learning-analytics/\u003c/li\u003e\n\u003cli\u003eSpinuzzi, C. (2005). The Methodology of Participatory Design. \u003cem\u003eTechnical Communication\u003c/em\u003e, 52, 163\u0026ndash;174.\u003c/li\u003e\n\u003cli\u003eVerbert, K., Duval, E., Klerkx, J., Govaerts, S., \u0026amp; Santos, J. L. (2013). Learning Analytics Dashboard Applications. \u003cem\u003eAmerican Behavioral Scientist\u003c/em\u003e, \u003cem\u003e57\u003c/em\u003e(10), 1500\u0026ndash;1509. https://doi.org/10.1177/0002764213479363\u003c/li\u003e\n\u003cli\u003eVerbert, K., Duval, E., Santos, J. L., Van Assche, F., Parra, G., Klerkx, J., \u0026amp; Govaerts, S. (2013). Learning dashboards: An overview and future research opportunities. \u003cem\u003ePersonal and Ubiquitous\u003c/em\u003e \u003cem\u003eComputing\u003c/em\u003e, \u003cem\u003e18\u003c/em\u003e, 1499\u0026ndash;1514. https://doi.org/10.1007/s00779-013-0751-2\u003c/li\u003e\n\u003cli\u003eVieira, C., Parsons, P., \u0026amp; Byrd, V. (2018). Visual learning analytics of educational data: A systematic literature review and research agenda. \u003cem\u003eComputers \u0026amp; Education\u003c/em\u003e, 122, 119\u0026ndash;135. https://doi.org/10.1016/j.compedu.2018.03.018\u003c/li\u003e\n\u003cli\u003eVillagr\u0026aacute;n, I., Hern\u0026aacute;ndez, R., Schuit, G., Neyem, A., Fuentes, J., Larrondo, L., Margozzini, E., Hurtado, M. T., Iriarte, Z., Miranda, C., Varas, J., \u0026amp; Hilliger, I. (2024). Enhancing Feedback Uptake and Self-Regulated Learning in Procedural Skills Training: Design and Evaluation of a Learning Analytics Dashboard. \u003cem\u003eJournal of Learning Analytics\u003c/em\u003e, \u003cem\u003e11\u003c/em\u003e(2), 138\u0026ndash;156. https://doi.org/10.18608/jla.2024.8195\u003c/li\u003e\n\u003cli\u003eWiley, K., Dimitriadis, Y., \u0026amp; Linn, M. (2024). A human-centred learning analytics approach for developing contextually scalable K‐12 teacher dashboards. \u003cem\u003eBritish Journal of Educational Technology\u003c/em\u003e, 55(3), 845\u0026ndash;885. https://doi.org/10.1111/bjet.13383\u003c/li\u003e\n\u003cli\u003eWise, A. F., Knight, S., \u0026amp; Ochoa, X. (2021). What Makes Learning Analytics Research Matter. \u003cem\u003eJournal of Learning Analytics\u003c/em\u003e, 8(3), 1\u0026ndash;9. https://doi.org/10.18608/jla.2021.7647\u003c/li\u003e\n\u003cli\u003eWong, B. T., \u0026amp; Li, K. C. (2020). A review of learning analytics intervention in higher education (2011\u0026ndash;2018). \u003cem\u003eJournal of Computers in Education\u003c/em\u003e, 7(1), 7\u0026ndash;28. https://doi.org/10.1007/s40692-019-00143-7\u003c/li\u003e\n\u003cli\u003eWorsley, M., Anderson, K., Melo, N., \u0026amp; Jang, J. (2021). Designing Analytics for Collaboration Literacy and Student Empowerment. \u003cem\u003eJournal of Learning Analytics\u003c/em\u003e, \u003cem\u003e8\u003c/em\u003e(1), 30\u0026ndash;48. https://doi.org/10.18608/jla.2021.7242\u003c/li\u003e\n\u003cli\u003eYan, L., Echeverria, V., Jin, Y., Fernandez‐Nieto, G., Zhao, L., Li, X., ... \u0026amp; Martinez‐Maldonado, R. (2024). Evidence‐based multimodal learning analytics for feedback and reflection in collaborative learning. \u003cem\u003eBritish Journal of Educational Technology\u003c/em\u003e, 55(5), 1900-1925. https://doi.org/10.1111/bjet.13498\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Footnotes","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003e SMCP are a set of pre-defined English phrases used in maritime communication to ensure clear and unambiguous messaging, especially in situations where language barriers may exist. These phrases are part of the International Maritime Organization (IMO) standards.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003e GMDSS is an internationally agreed-upon set of safety procedures, equipment, and communication protocols designed to enhance maritime safety and improve search and rescue operations for ships, boats, and aircraft in distress.\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003e COLREGs, or the International Regulations for Preventing Collisions at Sea, is a set of rules that govern the behaviour of vessels at sea to avoid collisions.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Human-Centred Learning Analytics (HCLA), Learning Analytics Dashboard (LAD), Educational Data Storytelling (EDS), Simulation-based training, Professional learning","lastPublishedDoi":"10.21203/rs.3.rs-7408850/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7408850/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThis study contributes to the growing body of research on Human-Centred Learning Analytics (HCLA) by exploring how to close the feedback loop between performance and reflection in simulation-based professional learning. Specifically, we investigate how student-facing learning analytics dashboards (LADs) can be designed to support sensemaking through contextualized data presentation, and what kinds of feedback and narrative elements enhance students\u0026rsquo; reflective engagement with LADs. Using user-centred design methods and Educational Data Storytelling (EDS), we developed a prototype that integrates HCLA principles with narrative feedback aligned with the educational organization of simulation-based training. The findings demonstrate how explanatory, student-facing dashboards can scaffold interpretation and promote reflection in simulation-based training. The study highlights the importance of moving beyond descriptive analytics toward explanatory designs that actively support student engagement. Future work should involve iterative testing with students and explore how such dashboards can be customized for simulations in different safety-critical domains.\u003c/p\u003e","manuscriptTitle":"Closing the Learning Analytics Loop through Explanatory Dashboard Design","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-08-20 07:19:08","doi":"10.21203/rs.3.rs-7408850/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"7a63c148-40bf-4066-8de6-72f672d904c5","owner":[],"postedDate":"August 20th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":53387263,"name":"Educational Philosophy and Theory"}],"tags":[],"updatedAt":"2025-08-20T07:19:08+00:00","versionOfRecord":[],"versionCreatedAt":"2025-08-20 07:19:08","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7408850","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7408850","identity":"rs-7408850","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}