A Cross-Sectional Pilot Audit of Accessible University Generative-AI Policies Using a Policy Clarity and Implementation Index

A Cross-Sectional Pilot Audit of Accessible University Generative-AI Policies Using a Policy Clarity and Implementation Index | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Cross-Sectional Pilot Audit of Accessible University Generative-AI Policies Using a Policy Clarity and Implementation Index Goutam Adwant This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8876294/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background: Universities have rapidly issued generative-AI (GenAI) guidance, but policy implementation detail remains uneven. Most existing analyses are descriptive, and reproducible instrument-based audits are scarce. Methods: Cross-sectional pilot audit (February 2026). Sampling frame: 500 institutions; operational crawl covered the first tranche under live-web/rate-limit constraints. We collected 81 policy pages and analysed 71 (≥200 words). Policies were scored with the rule-based Policy Clarity and Implementation Index (PCII; 12 indicators, 5 dimensions). Validation included inter-rater reliability (two trained coders, 22-policy subsample) and convergent validity (four expert raters scoring 15 policies on implementation specificity, correlated with PCII).We added coverage/nonresponse diagnostics and missingness scenario stress tests. Results: Coverage was highly uneven: 69.2% of the 500-frame was not attempted in this run, and analysed cases were concentrated in ranks 1–200. Composite PCII in the analysed sample was low (mean = 12.4; median = 8.3; range 0–62.5), with Assessment Guidance weakest (mean = 0.9/20; 77.5% scored 0). Interrater reliability showed strong composite agreement (ICC = 0.95); convergent validity with expert ratings yielded Spearman ρ = 0.83 (p < .001). Missingness scenarios showed the full-frame mean would remain below 20 unless unobserved institutions averaged above 21.3 points. Conclusions: This pilot provides preliminary evidence of a principles– procedures gap in public HTML-accessible GenAI policies, especially in assessment guidance, with additional gaps in privacy/data practice and accountability. Findings describe the analysed accessibility-constrained corpus and should not be generalized to institutional governance practices overall, which may rely on unpublished resources. Broader generalisation requires improved coverage. The validated index and open archive (DOI: [Anonymized for Review]) support replication and extension. Generative AI Higher education policy Policy analysis Assessment integrity Educational technology Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8876294","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":592224254,"identity":"d63268dc-46de-45c6-9986-28a7c82d8370","order_by":0,"name":"Goutam Adwant","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA+UlEQVRIiWNgGAWjYHACZgYGA4YEKMcGiBkbDxChxQCkhbGBgSENTBOhhQGu5TBYCK8Wg+O9j415Cv7kybv3mD/4mXPebm37YaAtNTbROLWcOW6czGNgUGx45oxhY++228nbziQCtRxLy23AocXsRhrz4RwDg8SNM3IMG3iBWswOALUwNhwmTkvj323nks3OPySsJRmkZb5EjmEz77YDdmY3CNhif+YYs/EfA+PEDTzHCmfLbktOMLsBtCUBj18k29uYJWf8kUuc39684ePbbXb2ZufTHz74UGODUwscGByA0IlglQmElIOAPNRQe2IUj4JRMApGwcgCAEOZZ4pS6gj/AAAAAElFTkSuQmCC","orcid":"","institution":"Independent Researcher","correspondingAuthor":true,"prefix":"","firstName":"Goutam","middleName":"","lastName":"Adwant","suffix":""}],"badges":[],"createdAt":"2026-02-14 02:23:48","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8876294/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8876294/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":108480785,"identity":"50a9a7f0-2a7e-4d01-8e60-c90934bb7201","added_by":"auto","created_at":"2026-05-05 07:56:01","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":610443,"visible":true,"origin":"","legend":"","description":"","filename":"anonymizedmanuscriptsubmission.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8876294/v1_covered_1f178ac1-69d7-4bb4-9d93-53ec1040ba98.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A Cross-Sectional Pilot Audit of Accessible University Generative-AI Policies Using a Policy Clarity and Implementation Index","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Generative AI, Higher education policy, Policy analysis, Assessment integrity, Educational technology","lastPublishedDoi":"10.21203/rs.3.rs-8876294/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8876294/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eBackground: Universities have rapidly issued generative-AI (GenAI) guidance, but policy implementation detail remains uneven. Most existing analyses are descriptive, and reproducible instrument-based audits are scarce.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eMethods: Cross-sectional pilot audit (February 2026). Sampling frame: 500 institutions; operational crawl covered the first tranche under live-web/rate-limit constraints. We collected 81 policy pages and analysed 71 (≥200 words). Policies were scored with the rule-based Policy Clarity and Implementation Index (PCII; 12 indicators, 5 dimensions). Validation included inter-rater reliability (two trained coders, 22-policy subsample) and convergent validity (four expert raters scoring 15 policies on implementation specificity, correlated with PCII).We added coverage/nonresponse diagnostics and missingness scenario stress tests.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eResults: Coverage was highly uneven: 69.2% of the 500-frame was not attempted in this run, and analysed cases were concentrated in ranks 1–200. Composite PCII in the analysed sample was low (mean = 12.4; median = 8.3; range 0–62.5), with Assessment Guidance weakest (mean = 0.9/20; 77.5% scored 0). Interrater reliability showed strong composite agreement (ICC = 0.95); convergent validity with expert ratings yielded Spearman ρ = 0.83 (p \u0026lt; .001). Missingness scenarios showed the full-frame mean would remain below 20 unless unobserved institutions averaged above 21.3 points.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eConclusions: This pilot provides preliminary evidence of a principles– procedures gap in public HTML-accessible GenAI policies, especially in assessment guidance, with additional gaps in privacy/data practice and accountability. Findings describe the analysed accessibility-constrained corpus and should not be generalized to institutional governance practices overall, which may rely on unpublished resources. Broader generalisation requires improved coverage. The validated index and open archive (DOI: [Anonymized for Review]) support replication and extension.\u003c/p\u003e","manuscriptTitle":"A Cross-Sectional Pilot Audit of Accessible University Generative-AI Policies Using a Policy Clarity and Implementation Index","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-05-05 07:55:31","doi":"10.21203/rs.3.rs-8876294/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"d8b852f2-8ccd-4f16-85d2-97232fdd4e5e","owner":[],"postedDate":"May 5th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-05-05T07:55:31+00:00","versionOfRecord":[],"versionCreatedAt":"2026-05-05 07:55:31","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8876294","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8876294","identity":"rs-8876294","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}