ARIA-QA: AI-Agent based Requirements Inspection and Analysis through Question Answering

ARIA-QA: AI-Agent based Requirements Inspection and Analysis through Question Answering | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article ARIA-QA: AI-Agent based Requirements Inspection and Analysis through Question Answering Chitrak Biswas, Souvick Das This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4399368/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 21 Oct, 2024 Read the published version in Innovations in Systems and Software Engineering → Version 1 posted 11 You are reading this latest preprint version Abstract Due to their predominant use of natural language (NL), requirements are prone to defects like inconsistency and incompleteness. Consequently, quality assurance processes are commonly applied to requirements manually. However, manual execution of these processes can be laborious and may inadvertently overlook critical quality issues due to time and budget constraints. This paper introduces ARIA, an innovative question-answering (QA) approach designed to automate support for stakeholders, including requirements engineers, during the analysis of NL requirements. The ability to pose questions and receive instant answers proves invaluable in proves beneficial in numerous quality-assurance scenarios, particularly in detecting incompleteness. The challenge of automating the answering of requirements-related questions is considerable, given the potential scope of the search for answers extending beyond the provided requirements specification. To overcome this challenge, ARIA integrates support for mining external domain knowledge resources like internet search results. Evaluation on seven diverse use cases drawn from the PURE dataset demonstrates ARIA's robustness and applicability across a range of real-life scenarios, highlighting its potential to significantly improve the quality and effectiveness of requirements analysis processes. This work represents one of the initial endeavors to seamlessly blend QA and external domain knowledge, effectively addressing complexities in requirements engineering. AI Agent Semantic Search Knowledge Graph Question Answering Retrieval Augmented generation Prompt Engineering Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 21 Oct, 2024 Read the published version in Innovations in Systems and Software Engineering → Version 1 posted Editorial decision: Revision requested 13 Jul, 2024 Reviews received at journal 26 Jun, 2024 Reviewers agreed at journal 17 Jun, 2024 Reviews received at journal 16 Jun, 2024 Reviews received at journal 11 Jun, 2024 Reviewers agreed at journal 05 Jun, 2024 Reviewers agreed at journal 05 Jun, 2024 Reviewers invited by journal 05 Jun, 2024 Editor assigned by journal 27 May, 2024 Submission checks completed at journal 11 May, 2024 First submitted to journal 10 May, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4399368","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":301333806,"identity":"bc5ad9e3-1ad3-40e1-b15d-84d68990d769","order_by":0,"name":"Chitrak Biswas","email":"","orcid":"","institution":"Amity University","correspondingAuthor":false,"prefix":"","firstName":"Chitrak","middleName":"","lastName":"Biswas","suffix":""},{"id":301333807,"identity":"1a8ca06e-8d38-4b2c-bc80-b7720ad67999","order_by":1,"name":"Souvick Das","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA30lEQVRIiWNgGAWjYLCCBCBkOMDA+ICBQQLIZWwgWguzAVRLIxF6IFrYJKA8/NbIt/eYSTyoSWPgO378WTXvDovEfgbm9gf4tBicOWMmkXAsh0HyTI7Zbd4zEokzGwg4zEAix9ggsaGCweBADttt3jYJY4MDBLTIz4BpOf/8WTFIiz0hLQw3cgwfJDbkMBjcSDBjBmqRMyAUYgZnjhU+SDiWxiN5442x5FygFonDjI0z8DqsvXnDwR81yXJ859MffnjbVsfD397+4ANeh0EBD4LJTIz6UTAKRsEoGAV4AQAwOUkLJ8whWwAAAABJRU5ErkJggg==","orcid":"","institution":"Ca' Foscari University of Venice","correspondingAuthor":true,"prefix":"","firstName":"Souvick","middleName":"","lastName":"Das","suffix":""}],"badges":[],"createdAt":"2024-05-10 08:38:24","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4399368/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4399368/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s11334-024-00589-8","type":"published","date":"2024-10-21T15:58:02+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":67681993,"identity":"1855d92e-52dc-44ab-99b0-5d91505169b3","added_by":"auto","created_at":"2024-10-28 16:12:14","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":327188,"visible":true,"origin":"","legend":"","description":"","filename":"ARIAQAAIAgentbasedRequirementsInspectionandAnalysisthroughQuestionAnswering.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4399368/v1_covered_2a178fe5-fb3c-48cd-a343-3ae1707e198c.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"ARIA-QA: AI-Agent based Requirements Inspection and Analysis through Question Answering","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"innovations-in-systems-and-software-engineering","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"isse","sideBox":"Learn more about [Innovations in Systems and Software Engineering](http://link.springer.com/journal/11334)","snPcode":"11334","submissionUrl":"https://submission.nature.com/new-submission/11334/3","title":"Innovations in Systems and Software Engineering","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"AI Agent, Semantic Search, Knowledge Graph, Question Answering, Retrieval Augmented generation, Prompt Engineering","lastPublishedDoi":"10.21203/rs.3.rs-4399368/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4399368/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Due to their predominant use of natural language (NL), requirements are prone to defects like inconsistency and incompleteness. Consequently, quality assurance processes are commonly applied to requirements manually. However, manual execution of these processes can be laborious and may inadvertently overlook critical quality issues due to time and budget constraints. This paper introduces ARIA, an innovative question-answering (QA) approach designed to automate support for stakeholders, including requirements engineers, during the analysis of NL requirements. The ability to pose questions and receive instant answers proves invaluable in proves beneficial in numerous quality-assurance scenarios, particularly in detecting incompleteness. The challenge of automating the answering of requirements-related questions is considerable, given the potential scope of the search for answers extending beyond the provided requirements specification. To overcome this challenge, ARIA integrates support for mining external domain knowledge resources like internet search results. Evaluation on seven diverse use cases drawn from the PURE dataset demonstrates ARIA's robustness and applicability across a range of real-life scenarios, highlighting its potential to significantly improve the quality and effectiveness of requirements analysis processes. This work represents one of the initial endeavors to seamlessly blend QA and external domain knowledge, effectively addressing complexities in requirements engineering.","manuscriptTitle":"ARIA-QA: AI-Agent based Requirements Inspection and Analysis through Question Answering","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-05-21 17:08:50","doi":"10.21203/rs.3.rs-4399368/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-07-13T09:29:36+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-06-26T13:35:41+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"119138405548696185751478819671746497377","date":"2024-06-17T13:23:05+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-06-16T08:50:19+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-06-11T10:29:25+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"4612456003651663065159845099816807367","date":"2024-06-05T09:12:49+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"266556104356893776754885941501203971631","date":"2024-06-05T06:56:45+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-06-05T06:48:33+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-05-27T15:12:00+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-05-11T12:04:51+00:00","index":"","fulltext":""},{"type":"submitted","content":"Innovations in Systems and Software Engineering","date":"2024-05-10T08:37:14+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"innovations-in-systems-and-software-engineering","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"isse","sideBox":"Learn more about [Innovations in Systems and Software Engineering](http://link.springer.com/journal/11334)","snPcode":"11334","submissionUrl":"https://submission.nature.com/new-submission/11334/3","title":"Innovations in Systems and Software Engineering","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"Springer Hybrid","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"3cf252f9-5916-4acc-9ed1-1d86cbbb2a14","owner":[],"postedDate":"May 21st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2024-10-28T16:04:28+00:00","versionOfRecord":{"articleIdentity":"rs-4399368","link":"https://doi.org/10.1007/s11334-024-00589-8","journal":{"identity":"innovations-in-systems-and-software-engineering","isVorOnly":false,"title":"Innovations in Systems and Software Engineering"},"publishedOn":"2024-10-21 15:58:02","publishedOnDateReadable":"October 21st, 2024"},"versionCreatedAt":"2024-05-21 17:08:50","video":"","vorDoi":"10.1007/s11334-024-00589-8","vorDoiUrl":"https://doi.org/10.1007/s11334-024-00589-8","workflowStages":[]},"version":"v1","identity":"rs-4399368","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4399368","identity":"rs-4399368","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}