Measuring the Unseen: A Situational Awareness Scoring Framework for Cyber Range Training

Measuring the Unseen: A Situational Awareness Scoring Framework for Cyber Range Training | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Measuring the Unseen: A Situational Awareness Scoring Framework for Cyber Range Training Amalia Damianou, Mariana S. Mazi, George Rizos, Konstantinos Bezas, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7997125/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 17 Mar, 2026 Read the published version in International Journal of Information Security → Version 1 posted 9 You are reading this latest preprint version Abstract As cyber-attacks grow in scale, sophistication, and impact, advanced training and evaluation platforms are essential for cybersecurity readiness. Cyber Ranges (CRs) provide controlled, high-fidelity simulation environments in which professionals can practice defensive strategies under realistic threat scenarios. In such environments, success depends not only on technical skill but also on robust Situational Awareness (SA), the cognitive capacity to perceive events, understand their meaning, and project future developments. Although SA is recognised as critical, existing CR evaluation methods typically focus on overall performance metrics (e.g., task completion rates, response accuracy, mission success). Such assessments often fail to capture the deeper, cognitive aspects of SA, limiting the CR’s ability to evaluate decision-making agility and strategic insight.In response, this paper presents the Situational Awareness Scoring System (SASS), a dynamic, multi-dimensional framework designed to quantify SA in live CR environments. SASS decomposes SA into core dimensions, perception, comprehension, projection, and resolution, then monitors trainee behaviour and scenario interactions in real time. Each dimension is assessed via weighted metrics: timing of responses, sequence and number of reasoning steps, usage and effectiveness of tools and techniques, hints requested, and decision outcomes. These metrics are aggregated into interpretable scores that reflect both aspects, specific performance and overall SA proficiency. In this study, we validate SASS in a series of cyber-defence exercises involving participants of differing expertise levels. Our results demonstrate that SASS scores correlate strongly with expert evaluations and can discriminate between high and low SA performers with high fidelity. SASS enables instructors to pinpoint cognitive weak spots (e.g. delayed perception or faulty projection) and adapt training on the fly. Ultimately, integrating SASS into CR platforms enriches the evaluation process, promoting not just technical competence but also the situational agility needed to counter modern cyber threats. Situational Awareness Cyber Range Digital Twin Scoring System Cybersecurity Training Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 17 Mar, 2026 Read the published version in International Journal of Information Security → Version 1 posted Editorial decision: Revision requested 06 Dec, 2025 Reviews received at journal 30 Nov, 2025 Reviewers agreed at journal 14 Nov, 2025 Reviews received at journal 11 Nov, 2025 Reviewers agreed at journal 10 Nov, 2025 Reviewers invited by journal 10 Nov, 2025 Editor assigned by journal 06 Nov, 2025 Submission checks completed at journal 06 Nov, 2025 First submitted to journal 31 Oct, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7997125","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":544957631,"identity":"81025f2c-1cfa-4e8c-9075-028fdf0d591f","order_by":0,"name":"Amalia Damianou","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAvElEQVRIiWNgGAWjYFCCBDBiYGBvbIAwiNfCc7CxgXgtYCCRwNhAlAb+9uSnGx7U3JEzuPm4/cEDhtp8glokzjwzu5Fw7Jmxwe1EkMOOWxK26kYCUAvb4cQNEC3HDAjqkL+R/u1Gwj+glpsHidRicCPH7EZiG1DLDUaQlhrCWgzPvCm7kdh32FjyTGLjjASDA4S1yB1P33bzx7fDcnzHjz/4+KOijrAWdHceJlUHA0Md6VpGwSgYBaNg2AMAzA1J/RggxVUAAAAASUVORK5CYII=","orcid":"","institution":"Centre for Research and Technology Hellas","correspondingAuthor":true,"prefix":"","firstName":"Amalia","middleName":"","lastName":"Damianou","suffix":""},{"id":544957632,"identity":"ac18db83-2dfc-4c3f-a5fc-370b108d7ae1","order_by":1,"name":"Mariana S. Mazi","email":"","orcid":"","institution":"Centre for Research and Technology Hellas","correspondingAuthor":false,"prefix":"","firstName":"Mariana","middleName":"S.","lastName":"Mazi","suffix":""},{"id":544957633,"identity":"81379fd9-08c4-4830-a464-320d5ec9752f","order_by":2,"name":"George Rizos","email":"","orcid":"","institution":"Centre for Research and Technology Hellas","correspondingAuthor":false,"prefix":"","firstName":"George","middleName":"","lastName":"Rizos","suffix":""},{"id":544957634,"identity":"351406b3-b3cd-4414-bbc1-6ba8007f14cd","order_by":3,"name":"Konstantinos Bezas","email":"","orcid":"","institution":"Centre for Research and Technology Hellas","correspondingAuthor":false,"prefix":"","firstName":"Konstantinos","middleName":"","lastName":"Bezas","suffix":""},{"id":544957635,"identity":"58cf2a16-326b-4533-ab7b-664ed0d750ec","order_by":4,"name":"George Lazaridis","email":"","orcid":"","institution":"Centre for Research and Technology Hellas","correspondingAuthor":false,"prefix":"","firstName":"George","middleName":"","lastName":"Lazaridis","suffix":""},{"id":544957636,"identity":"3e70026e-20e1-4842-bdaf-289dbcea49ee","order_by":5,"name":"Antonis Voulgaridis","email":"","orcid":"","institution":"Centre for Research and Technology Hellas","correspondingAuthor":false,"prefix":"","firstName":"Antonis","middleName":"","lastName":"Voulgaridis","suffix":""},{"id":544957637,"identity":"dfe28b45-1ba0-4e65-a730-11f3ea05704a","order_by":6,"name":"Konstantinos Votis","email":"","orcid":"","institution":"Centre for Research and Technology Hellas","correspondingAuthor":false,"prefix":"","firstName":"Konstantinos","middleName":"","lastName":"Votis","suffix":""},{"id":544957638,"identity":"a0074b19-b74b-461e-99c6-52d1f265e480","order_by":7,"name":"Dimitrios Tzovaras","email":"","orcid":"","institution":"Centre for Research and Technology Hellas","correspondingAuthor":false,"prefix":"","firstName":"Dimitrios","middleName":"","lastName":"Tzovaras","suffix":""}],"badges":[],"createdAt":"2025-10-31 09:53:41","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7997125/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7997125/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s10207-026-01219-z","type":"published","date":"2026-03-17T15:59:14+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":96259598,"identity":"2f3beaad-b498-42d0-a469-4ac1e2b0edad","added_by":"auto","created_at":"2025-11-19 07:55:57","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":9700,"visible":true,"origin":"","legend":"","description":"","filename":"4a4e715ff2a74cd1aaa213d278dd11f9.json","url":"https://assets-eu.researchsquare.com/files/rs-7997125/v1/7cae640a837b64791aae44fb.json"},{"id":105224185,"identity":"c1e35f08-f512-4f6d-9036-c2f88b6d07c6","added_by":"auto","created_at":"2026-03-23 16:13:02","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":949538,"visible":true,"origin":"","legend":"","description":"","filename":"MeasuringtheUnseenASituationalAwarenessScoringFrameworkforCyberRangeTraining.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7997125/v1_covered_cdee4ae1-4ef8-4e5c-95b7-b781a7b74a96.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Measuring the Unseen: A Situational Awareness Scoring Framework for Cyber Range Training","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"international-journal-of-information-security","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ijis","sideBox":"Learn more about [International Journal of Information Security](http://link.springer.com/journal/10207)","snPcode":"10207","submissionUrl":"https://submission.nature.com/new-submission/10207/3","title":"International Journal of Information Security","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"Situational Awareness, Cyber Range, Digital Twin, Scoring System, Cybersecurity Training","lastPublishedDoi":"10.21203/rs.3.rs-7997125/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7997125/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"As cyber-attacks grow in scale, sophistication, and impact, advanced training and evaluation platforms are essential for cybersecurity readiness. Cyber Ranges (CRs) provide controlled, high-fidelity simulation environments in which professionals can practice defensive strategies under realistic threat scenarios. In such environments, success depends not only on technical skill but also on robust Situational Awareness (SA), the cognitive capacity to perceive events, understand their meaning, and project future developments. Although SA is recognised as critical, existing CR evaluation methods typically focus on overall performance metrics (e.g., task completion rates, response accuracy, mission success). Such assessments often fail to capture the deeper, cognitive aspects of SA, limiting the CR’s ability to evaluate decision-making agility and strategic insight.In response, this paper presents the Situational Awareness Scoring System (SASS), a dynamic, multi-dimensional framework designed to quantify SA in live CR environments. SASS decomposes SA into core dimensions, perception, comprehension, projection, and resolution, then monitors trainee behaviour and scenario interactions in real time. Each dimension is assessed via weighted metrics: timing of responses, sequence and number of reasoning steps, usage and effectiveness of tools and techniques, hints requested, and decision outcomes. These metrics are aggregated into interpretable scores that reflect both aspects, specific performance and overall SA proficiency. In this study, we validate SASS in a series of cyber-defence exercises involving participants of differing expertise levels. Our results demonstrate that SASS scores correlate strongly with expert evaluations and can discriminate between high and low SA performers with high fidelity. SASS enables instructors to pinpoint cognitive weak spots (e.g. delayed perception or faulty projection) and adapt training on the fly. Ultimately, integrating SASS into CR platforms enriches the evaluation process, promoting not just technical competence but also the situational agility needed to counter modern cyber threats.","manuscriptTitle":"Measuring the Unseen: A Situational Awareness Scoring Framework for Cyber Range Training","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-11-19 07:43:29","doi":"10.21203/rs.3.rs-7997125/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-12-06T07:03:35+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-30T12:03:31+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"46579639015889260010006838858416408085","date":"2025-11-14T08:13:45+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-11T14:48:40+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"225443314898685999243553019515127207049","date":"2025-11-10T17:33:05+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-11-10T10:07:04+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-11-07T04:44:08+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-11-06T16:32:01+00:00","index":"","fulltext":""},{"type":"submitted","content":"International Journal of Information Security","date":"2025-10-31T09:50:27+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"international-journal-of-information-security","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"ijis","sideBox":"Learn more about [International Journal of Information Security](http://link.springer.com/journal/10207)","snPcode":"10207","submissionUrl":"https://submission.nature.com/new-submission/10207/3","title":"International Journal of Information Security","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"e72ab953-7964-4c70-b8b7-7b2e097b5cbb","owner":[],"postedDate":"November 19th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2026-03-23T16:07:42+00:00","versionOfRecord":{"articleIdentity":"rs-7997125","link":"https://doi.org/10.1007/s10207-026-01219-z","journal":{"identity":"international-journal-of-information-security","isVorOnly":false,"title":"International Journal of Information Security"},"publishedOn":"2026-03-17 15:59:14","publishedOnDateReadable":"March 17th, 2026"},"versionCreatedAt":"2025-11-19 07:43:29","video":"","vorDoi":"10.1007/s10207-026-01219-z","vorDoiUrl":"https://doi.org/10.1007/s10207-026-01219-z","workflowStages":[]},"version":"v1","identity":"rs-7997125","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7997125","identity":"rs-7997125","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}