Automatic Assessment of UML Models Based on Neural Networks for Students in a Software Engineering Course

Automatic Assessment of UML Models Based on Neural Networks for Students in a Software Engineering Course | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Method Article Automatic Assessment of UML Models Based on Neural Networks for Students in a Software Engineering Course Irina-Gabriela Nedelcu, Stefan Alexandru Mocanu, Daniela Saru, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9428571/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The impact of artificial intelligence on educational practices has been studied intensively, from searching to generating content and ethical issues. This article investigates its application for educational purposes, i.e., to automatically assess software models realized by students and to detect incorrect completion of practical work if that is the case; this may be useful to teachers or even to students themselves. Our automated assessment software does a classification of models conform to the standard Unified Modeling Language (UML), using neural networks trained on public datasets and on more than eight hundred models collected from our university students; thus, starting from models provided by students as images, it distinguishes between various types of UML diagrams, and it also detects whether they do not conform to the standard language. The investigation includes a detailed analysis performed for three cohorts totaling 237 students, to identify correct assessments, recurring errors, and the impact of diagram quality, missing elements, duplicates, and non-standard notations. To perform an evaluation under imbalanced data conditions, macro average metrics were applied, including precision, recall, specificity, and F1 score. The assessment software results show that the assessment performs reliably on structured UML diagrams and generalizes well to real-world student submissions, with a macro average F1 score of 0.96. These findings demonstrate the feasibility of AI-based assessment of students’ UML modeling practical work and highlight opportunities for providing them with automated feedback and guidance. artificial intelligence in education automated assessment software models neural networks image processing Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9428571","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Method Article","associatedPublications":[],"authors":[{"id":623746860,"identity":"d334a8f5-e785-4ad3-b9a1-d64aa5762a0e","order_by":0,"name":"Irina-Gabriela Nedelcu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABHUlEQVRIie2RsWrDMBBATwjcxXZXCbXNLyhkKvRjLPoTGYrrYFCXZI/B9BsyJWSzEdSLP8BDocmSKYOhi6EeKjUtpWDijIHqTafTPXS6A7BYzhAHUHSIMqyDsYmC7zvSq5igNHU9yi9GQfIExWdKbhp4vfGLyeTdew5DXgSobloQKxY5dd3RmC+ehlPYjWiZx8xbKsLLDaYzCWKdZjiZdyguksQFJRaVkNhbZoRXATDdpM4EGLvdCm1BPS7etlpJQ6PgD2iPK0y/EvAKaSXCRnGYnuRx5YqrYVKKmKYviiblVt7OJBmt0zxGHX8ZTC92dD9WA79Qeb1/CC/94l5VTXt3vWKxgo6JHeB/Tl/LJTr5s+WT4f0lFovF8j/4BMFpZpOczldQAAAAAElFTkSuQmCC","orcid":"","institution":"National University of Science and Technology POLITEHNICA Bucharest","correspondingAuthor":true,"prefix":"","firstName":"Irina-Gabriela","middleName":"","lastName":"Nedelcu","suffix":""},{"id":623746861,"identity":"e5f49dd7-f2eb-4a61-9d99-615d98a96ee7","order_by":1,"name":"Stefan Alexandru Mocanu","email":"","orcid":"","institution":"National University of Science and Technology POLITEHNICA Bucharest","correspondingAuthor":false,"prefix":"","firstName":"Stefan","middleName":"Alexandru","lastName":"Mocanu","suffix":""},{"id":623746862,"identity":"11b9ebf7-dfe9-4732-bfc4-4b7fd772a343","order_by":2,"name":"Daniela Saru","email":"","orcid":"","institution":"National University of Science and Technology POLITEHNICA Bucharest","correspondingAuthor":false,"prefix":"","firstName":"Daniela","middleName":"","lastName":"Saru","suffix":""},{"id":623746863,"identity":"28cd3748-7b5b-4905-a544-0cca8481a627","order_by":3,"name":"Elvira Popescu","email":"","orcid":"","institution":"University of Craiova","correspondingAuthor":false,"prefix":"","firstName":"Elvira","middleName":"","lastName":"Popescu","suffix":""},{"id":623746865,"identity":"170eeaf2-9c96-4c70-abac-55b18b5d1eaa","order_by":4,"name":"Simona Iuliana Caramihai","email":"","orcid":"","institution":"National University of Science and Technology POLITEHNICA Bucharest","correspondingAuthor":false,"prefix":"","firstName":"Simona","middleName":"Iuliana","lastName":"Caramihai","suffix":""},{"id":623746866,"identity":"5a61a987-7cbf-4b5c-9898-b9f86fa69165","order_by":5,"name":"Anca Daniela Ionita","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAzklEQVRIiWNgGAWjYNACAwYGNgYGxgcMDAdI0cLGwGxAghYQYGNgkyBKC//s5ocfGAps8vjkm59V8+64w2De3oBfi8SdY8YSDAZpxWxsbGa3ec88Y5A5Q8imGzkMQC2HE9vYGIBa2g4zSEgk4NchfyOH+QdEC/u3YqK0GNzIYYPawmPGTJQWwxtpZhYJBmlALTnFknPPPOOR4CHgF7kbyY9vfPhjkzi/+fjGD2933JGTYG/ArwUM4C5hbGDgIUI9MmAkxoJRMApGwSgYcQAALAQ9uY0H6ScAAAAASUVORK5CYII=","orcid":"","institution":"National University of Science and Technology POLITEHNICA Bucharest","correspondingAuthor":true,"prefix":"","firstName":"Anca","middleName":"Daniela","lastName":"Ionita","suffix":""}],"badges":[],"createdAt":"2026-04-15 15:14:03","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-9428571/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9428571/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":107481288,"identity":"3e30db59-6e8c-4216-b887-6ed766492486","added_by":"auto","created_at":"2026-04-22 02:16:59","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":390431,"visible":true,"origin":"","legend":"","description":"","filename":"ArticolINFinal.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9428571/v1_covered_4210e91c-0ede-49d3-9e2c-46d9ebe087aa.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eAutomatic Assessment of UML Models Based on Neural Networks for Students in a Software Engineering Course\u003c/strong\u003e\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"artificial intelligence in education, automated assessment, software models, neural networks, image processing","lastPublishedDoi":"10.21203/rs.3.rs-9428571/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9428571/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThe impact of artificial intelligence on educational practices has been studied intensively, from searching to generating content and ethical issues. This article investigates its application for educational purposes, i.e., to automatically assess software models realized by students and to detect incorrect completion of practical work if that is the case; this may be useful to teachers or even to students themselves. Our automated assessment software does a classification of models conform to the standard Unified Modeling Language (UML), using neural networks trained on public datasets and on more than eight hundred models collected from our university students; thus, starting from models provided by students as images, it distinguishes between various types of UML diagrams, and it also detects whether they do not conform to the standard language. The investigation includes a detailed analysis performed for three cohorts totaling 237 students, to identify correct assessments, recurring errors, and the impact of diagram quality, missing elements, duplicates, and non-standard notations. To perform an evaluation under imbalanced data conditions, macro average metrics were applied, including precision, recall, specificity, and F1 score. The assessment software results show that the assessment performs reliably on structured UML diagrams and generalizes well to real-world student submissions, with a macro average F1 score of 0.96. These findings demonstrate the feasibility of AI-based assessment of students\u0026rsquo; UML modeling practical work and highlight opportunities for providing them with automated feedback and guidance.\u003c/p\u003e","manuscriptTitle":"Automatic Assessment of UML Models Based on Neural Networks for Students in a Software Engineering Course","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-16 09:20:05","doi":"10.21203/rs.3.rs-9428571/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"43b991f5-d48c-45d5-a8bc-91b2bbbd2268","owner":[],"postedDate":"April 16th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-04-16T09:20:05+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-16 09:20:05","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9428571","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9428571","identity":"rs-9428571","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}