How AI-assisted problem solving dissociates competence and performance in higher education

How AI-assisted problem solving dissociates competence and performance in higher education | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article How AI-assisted problem solving dissociates competence and performance in higher education Trinché This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8036878/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract This paper introduces and quantifies the Hyperfocus Bias Index (HBI)—a narrowing of attentional and strategic focus toward a subset of often advanced items during problem-solving tasks assisted by large language models (LLMs). We conduct a large-scale simulation of item response theory (IRT)-calibrated multiple-choice assessments, comparing a no-assistance condition to two LLM-assisted conditions of differing reliability (Qwen2, Mistral-7B), to examine how AI reshapes effort allocation, decision sequences, and, ultimately, competence assessment. The HBI is operationalized through a composite index combining temporal concentration, clustering of AI calls, reduced post-error switching, advanced-item tunneling, and post-error slowing. Results show that highly reliable assistance markedly increases the HBI: time and queries become concentrated on a few tasks, extended advanced-item sequences emerge, and post-failure flexibility decreases. Crucially, hyperfocal profiles are associated with local gains (success on difficult items) but lower overall performance, suggesting a dissociation between internalized competence and assisted performance. We discuss pedagogical safeguards—transparency regarding the “optimal dose” of assistance, incentives for strategic switching, and calibration of AI support—to preserve the integrity of learning and assessment in the era of LLMs. Cognitive Neuroscience Human-AI Learning strategy Human behavior Higher education Full Text Additional Declarations The authors declare no competing interests. Supplementary Files episodes.csv episodeswithhbi.csv Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8036878","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":540311163,"identity":"8c9bfb86-09b1-4ff8-9d4d-11e32a54101c","order_by":0,"name":"Trinché","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABDUlEQVRIiWNgGAWjYDACZjApAUKMDxgYLGQY2GEiDIwN2HTwgLQcgGhhNgCSPAhDcGlhAGsBq2GTQNKCG9izcycwf9xjkcc/u/lYNU8FSAvzwc+VOyzy5B2YGx9gdRjvBoYDzySKJe4cS7vNcwakhS1Z8uwZiWLDA4zNBji1HJBI3CCRY3Y7t02Cx/4wj4FkY5tE4sYGxjYJ/FryvxXn/gPZwv/5J5FactiYcxtAWnjYwLbMZ8Ch5TDvhgNngFpm3Egzlv5zDOwXM0uQlg3M2P3C3n9244OKA3WJ/TOSH36cUWMjx8De/PhmY1td4vz29ofYQgwEDmAXNjiMQz1uIN9AspZRMApGwSgYngAAGwpaW5KX2MsAAAAASUVORK5CYII=","orcid":"https://orcid.org/0009-0009-8315-9650","institution":"Pragma Learning Institute","correspondingAuthor":true,"prefix":"","firstName":"","middleName":"","lastName":"Trinché","suffix":""}],"badges":[],"createdAt":"2025-11-05 09:43:06","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-8036878/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8036878/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":95256134,"identity":"b4d03c4b-8c51-4cf5-8aa4-6f25367d95c1","added_by":"auto","created_at":"2025-11-06 03:16:36","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":348835,"visible":true,"origin":"","legend":"","description":"","filename":"HowAIassistedproblemsolvingdissociatescompetenceandperformanceinhighereducation.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8036878/v1_covered_c67a040e-3a4a-47db-b2a9-e1c0b643dca8.pdf"},{"id":95256132,"identity":"628ce902-5cbc-46a3-b14e-d52de4ed70c5","added_by":"auto","created_at":"2025-11-06 03:16:31","extension":"csv","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":44079223,"visible":true,"origin":"","legend":"","description":"","filename":"episodes.csv","url":"https://assets-eu.researchsquare.com/files/rs-8036878/v1/d47684e5ce7241b4ca2ee222.csv"},{"id":95256133,"identity":"0d7dbe7d-9fec-49b7-afe6-b22be77f6e87","added_by":"auto","created_at":"2025-11-06 03:16:31","extension":"csv","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":51978168,"visible":true,"origin":"","legend":"","description":"","filename":"episodeswithhbi.csv","url":"https://assets-eu.researchsquare.com/files/rs-8036878/v1/e09cd55e30ebb5583702b9bf.csv"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eHow AI-assisted problem solving dissociates competence and performance in higher education\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Pragma Learning Institute","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Human-AI, Learning strategy, Human behavior, Higher education","lastPublishedDoi":"10.21203/rs.3.rs-8036878/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8036878/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eThis paper introduces and quantifies the \u003cem\u003eHyperfocus Bias Index\u003c/em\u003e (HBI)—a narrowing of attentional and strategic focus toward a subset of often advanced items during problem-solving tasks assisted by large language models (LLMs). We conduct a large-scale simulation of item response theory (IRT)-calibrated multiple-choice assessments, comparing a no-assistance condition to two LLM-assisted conditions of differing reliability (Qwen2, Mistral-7B), to examine how AI reshapes effort allocation, decision sequences, and, ultimately, competence assessment. The HBI is operationalized through a composite index combining temporal concentration, clustering of AI calls, reduced post-error switching, advanced-item tunneling, and post-error slowing. Results show that highly reliable assistance markedly increases the HBI: time and queries become concentrated on a few tasks, extended advanced-item sequences emerge, and post-failure flexibility decreases. Crucially, hyperfocal profiles are associated with local gains (success on difficult items) but lower overall performance, suggesting a dissociation between internalized competence and assisted performance. We discuss pedagogical safeguards—transparency regarding the “optimal dose” of assistance, incentives for strategic switching, and calibration of AI support—to preserve the integrity of learning and assessment in the era of LLMs.\u003c/p\u003e","manuscriptTitle":"How AI-assisted problem solving dissociates competence and performance in higher education","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-11-06 03:16:25","doi":"10.21203/rs.3.rs-8036878/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2b994a9d-b2e4-4231-a59b-fe3679a190a5","owner":[],"postedDate":"November 6th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":57519738,"name":"Cognitive Neuroscience"}],"tags":[],"updatedAt":"2025-11-06T03:16:26+00:00","versionOfRecord":[],"versionCreatedAt":"2025-11-06 03:16:25","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8036878","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8036878","identity":"rs-8036878","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}