Item-level monitoring, response style stability, and the hard-easy effect

Item-level monitoring, response style stability, and the hard-easy effect | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Item-level monitoring, response style stability, and the hard-easy effect Roy B. Clariana, Eunsung Park This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-256756/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 26 Mar, 2021 Read the published version in Educational Technology Research and Development → Version 1 posted You are reading this latest preprint version Abstract Cognitive and metacognitive processes during learning depend on accurate monitoring, this investigation examines the influence of immediate item-level knowledge of correct response feedback on cognition monitoring accuracy. In an optional end-of-course computer-based review lesson, participants (n = 68) were randomly assigned to groups to receive either immediate item-by-item feedback (IF) or no immediate feedback (NF). Item-by-item monitoring consisted of confidence self-reports. Two days later, participants completed a retention test (IF = NF, no significant difference). Monitoring accuracy during the review lesson was low, and contrary to expectations was significantly less with immediate feedback (IF < NF, Cohen’s d = .62). Descriptive data shows that (1) monitoring accuracy can be attributed to cues beyond actual item difficulty, (2) a hard-easy effect was observed where item difficulty was related to confidence judgements as a non-monotonic function, (3) response confidence was predicted by the Coh-Metrix dimension Word Concreteness in both the IF and NF treatments, and (4) significant autocorrelations (hysteresis) for confidence measures were observed for NF but not for IF. It seems likely that monitoring is based on multiple and sometimes competing cues, the salience of each relates in some degree to content difficulty, but that the stability of individual response styles plays a substantive role in monitoring. This investigation shows the need for new applications of technology for monitoring multiple measures on the fly to better understand SRL processes to support all learners. Educational Philosophy and Theory self-regulated learning response confidence response style stability hysteresis hard-easy effect Coh-Metrix Figures Figure 1 Figure 2 Figure 3 Full Text Declarations This human subjects research was conducted under an approved university IRB. The authors have no competing interest. Cite Share Download PDF Status: Published Journal Publication published 26 Mar, 2021 Read the published version in Educational Technology Research and Development → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-256756","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":12630983,"identity":"92014249-7322-4554-b20f-6dcf10a2cd3f","order_by":0,"name":"Roy B. Clariana","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1ElEQVRIiWNgGAWjYBAC+/bDDQwMBkAWewNYgLGBkBaDM4lQLTwHiNVyQxGqRiKBaC2MjZ8rCuzy+Ge+PfyZh8FGdsMBAlrsZzA2S54xSC6WuJ2XJs3DkGZMUIudBGODZIMBc2LD7RwzZh6Gw4kEtRhLMDb/bDCoT5x/84wx0GH/CWsxnKHYBrQFaPgNHgOgww4Q1gIM5DbLBoPjiRvP5KVJzjFINp5JUMvxw4dvNvypTpx3/OzhD28q7GT7CGlBAjwMkDglAfCQpnwUjIJRMApGDgAAQohHyNeBqK4AAAAASUVORK5CYII=","orcid":"https://orcid.org/0000-0001-9374-0064","institution":"Pennsylvania State University","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Roy","middleName":"B.","lastName":"Clariana","suffix":""},{"id":12630984,"identity":"2909324e-7531-40e0-a6ae-844ba323babc","order_by":1,"name":"Eunsung Park","email":"","orcid":"","institution":"Pennsylvania State University University Park : Penn State","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Eunsung","middleName":"","lastName":"Park","suffix":""}],"badges":[],"createdAt":"2021-02-18 21:38:31","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-256756/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-256756/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1007/s11423-021-09981-8","type":"published","date":"2021-03-26T19:10:37+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":6413512,"identity":"11a447cf-1c3d-4583-90b8-8ab8f281548f","added_by":"auto","created_at":"2021-02-26 21:59:25","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":75449,"visible":true,"origin":"","legend":"Group-level examination performance of five ability groupings (G1-G5) with predicted and postdicted performance judgements at two time periods (from Hacker at al., 2000).","description":"","filename":"1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-256756/v1/b4577042f37cc8f0a97faa09.jpg"},{"id":6413116,"identity":"2fc22ac6-bf83-4e44-9bc8-84d4a5e57cb2","added_by":"auto","created_at":"2021-02-26 21:56:25","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":45690,"visible":true,"origin":"","legend":"Scatterplots of the actual lesson performance average lesson item difficulties (item P) and item confidence for the NF and IF treatments.","description":"","filename":"2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-256756/v1/c8fcb112137477b1b6f930bd.jpg"},{"id":6413810,"identity":"22674529-53c6-4276-8f71-393ea07ba8c4","added_by":"auto","created_at":"2021-02-26 22:02:25","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":138821,"visible":true,"origin":"","legend":"IF (top) and NF (bottom) treatments’ review lesson average item-level difficulty (solid lines), item-level confidence (dashed lines).","description":"","filename":"3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-256756/v1/3512d11254acbb2f3716a29f.jpg"},{"id":13596008,"identity":"e7e04109-d72b-40fb-a768-14e474caafb7","added_by":"auto","created_at":"2021-09-17 05:26:44","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":598189,"visible":true,"origin":"","legend":"","description":"","filename":"ClarianafinalETRDD1900654Feb182021.pdf","url":"https://assets-eu.researchsquare.com/files/rs-256756/v1_covered.pdf"},{"id":6414029,"identity":"5310e38b-e27f-407d-9682-698291307e81","added_by":"auto","created_at":"2021-02-26 22:05:30","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":513000,"visible":true,"origin":"","legend":"","description":"","filename":"ClarianafinalETRDD1900654Feb182021.pdf","url":"https://assets-eu.researchsquare.com/files/rs-256756/v1_stamped.pdf"}],"financialInterests":"","formattedTitle":"Item-level monitoring, response style stability, and the hard-easy effect","fulltext":[{"header":"Full Text","content":"\u003cp\u003eThis preprint is available for \u003ca href='/article/rs-256756/latest.pdf' target='_blank'\u003edownload as a PDF\u003c/a\u003e.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003eThis human subjects research was conducted under an approved university IRB. The authors have no competing interest.\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"self-regulated learning, response confidence, response style stability, hysteresis, hard-easy effect, Coh-Metrix","lastPublishedDoi":"10.21203/rs.3.rs-256756/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-256756/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Cognitive and metacognitive processes during learning depend on accurate monitoring, this investigation examines the influence of immediate item-level knowledge of correct response feedback on cognition monitoring accuracy. In an optional end-of-course computer-based review lesson, participants (n = 68) were randomly assigned to groups to receive either immediate item-by-item feedback (IF) or no immediate feedback (NF). Item-by-item monitoring consisted of confidence self-reports. Two days later, participants completed a retention test (IF = NF, no significant difference). Monitoring accuracy during the review lesson was low, and contrary to expectations was significantly less with immediate feedback (IF \u0026lt; NF, Cohen’s d = .62). Descriptive data shows that (1) monitoring accuracy can be attributed to cues beyond actual item difficulty, (2) a hard-easy effect was observed where item difficulty was related to confidence judgements as a non-monotonic function, (3) response confidence was predicted by the Coh-Metrix dimension Word Concreteness in both the IF and NF treatments, and (4) significant autocorrelations (hysteresis) for confidence measures were observed for NF but not for IF. It seems likely that monitoring is based on multiple and sometimes competing cues, the salience of each relates in some degree to content difficulty, but that the stability of individual response styles plays a substantive role in monitoring. This investigation shows the need for new applications of technology for monitoring multiple measures on the fly to better understand SRL processes to support all learners.","manuscriptTitle":"Item-level monitoring, response style stability, and the hard-easy effect","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2021-02-26 21:56:23","doi":"10.21203/rs.3.rs-256756/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"9cd4b855-01f9-4e87-9164-1b234a723586","owner":[],"postedDate":"February 26th, 2021","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":2660263,"name":"Educational Philosophy and Theory"}],"tags":[],"updatedAt":"2021-08-18T19:31:11+00:00","versionOfRecord":{"articleIdentity":"rs-256756","link":"https://doi.org/10.1007/s11423-021-09981-8","journal":{"identity":"educational-technology-research-and-development","isVorOnly":false,"title":"Educational Technology Research and Development"},"publishedOn":"2021-03-26 19:10:37","publishedOnDateReadable":"March 26th, 2021"},"versionCreatedAt":"2021-02-26 21:56:23","video":"","vorDoi":"10.1007/s11423-021-09981-8","vorDoiUrl":"https://doi.org/10.1007/s11423-021-09981-8","workflowStages":[]},"version":"v1","identity":"rs-256756","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-256756","identity":"rs-256756","version":["v1"]},"buildId":"ApUGefWb6u5IBVtyqm6d5","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}