Inside the learners’ minds: Exploring response-process validity and diagnosing misconceptions in the Pharmacology Concept Inventory

Inside the learners’ minds: Exploring response-process validity and diagnosing misconceptions in the Pharmacology Concept Inventory | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Inside the learners’ minds: Exploring response-process validity and diagnosing misconceptions in the Pharmacology Concept Inventory Adeladlew K. Netere, Tony Hughes, Anna-Marie Babey, Willmann Liang, and 13 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9077666/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 6 You are reading this latest preprint version Abstract Evaluating pharmacology teaching and learning methods requires assessment tools with strong evidence of validity. The Pharmacology Concept Inventory (PCI) was developed to measure concept-based learning in pharmacology. To achieve this, evidence from learners’ perspectives must be collected alongside expert-guided refinement of the tool. This study aimed to gather evidence for the response process validity (RPV) of the PCI, identify new misconceptions, and highlight areas for improvement. We adopted a qualitative approach, drawing on multiple sources: analyzing students’ open-ended responses (OERs), conducting inductive thematic analysis of cognitive interview data, and involving experts in item review. Of the 70 students invited, about two-thirds submitted an OER, 16 participated in think-aloud cognitive interviews, and 5 experts contributed to item refinement. The OERs revealed common patterns of understanding, misconceptions, and uncertainty. The thematic analysis of cognitive interviews uncovered three main themes: interpretation and understanding of items; pharmacological reasoning and evidence use; and validation of conceptual understanding and distractor function, including the emergence of new misconceptions. These findings helped develop a cognitive diagnostic framework that provides evidence for RPV and allows systematic detection of emerging misconceptions in the PCI. The study clarified how learners interpret PCI items and how scientific explanations or misconceptions influence their decision-making. Correct, scientifically reasoned responses were linked with proper understanding, while distractor analysis highlighted recurring misconceptions and new alternative ideas. Overall, these results support strategies for collecting RPV evidence, identifying misconceptions, and refining assessment items to enhance concept-based pharmacology education. pharmacology concept inventory response process validity misconceptions cognitive interview Full Text Additional Declarations No competing interests reported. Supplementary Files Supplementaryfile1Table1DemographiccharacteristicsofrespondentsinthepilotsurveyN70.docx Supplementaryfile2Table2NumberofopenendedresponsesbyPCIquestion.docx Supplementaryfile3Table3Cognitiveinterviewprotocol.docx SupplementaryFile4Interviewguidethinkaloudandprobingapproach.docx Supplementaryfile5Figure1ReasoningdistributionacrosspharmacokineticsPCIitems.docx Supplementaryfile6Figure2ReasoningdistributionacrosspharmacodynamicsPCIitems.docx Cite Share Download PDF Status: Under Review Version 1 posted Reviewers agreed at journal 30 Apr, 2026 Reviewers invited by journal 17 Apr, 2026 Editor assigned by journal 13 Apr, 2026 Editor invited by journal 19 Mar, 2026 Submission checks completed at journal 18 Mar, 2026 First submitted to journal 17 Mar, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9077666","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":627767589,"identity":"1214fe7a-4e70-4775-af22-779bc69d86cb","order_by":0,"name":"Adeladlew K. Netere","email":"","orcid":"","institution":"Monash University","correspondingAuthor":false,"prefix":"","firstName":"Adeladlew","middleName":"K.","lastName":"Netere","suffix":""},{"id":627767592,"identity":"b77ee25f-1d4a-4013-8f0f-ec34ab526cb4","order_by":1,"name":"Tony Hughes","email":"","orcid":"","institution":"Monash University","correspondingAuthor":false,"prefix":"","firstName":"Tony","middleName":"","lastName":"Hughes","suffix":""},{"id":627767593,"identity":"9c0f0573-2b93-435f-bff5-563820d2b509","order_by":2,"name":"Anna-Marie Babey","email":"","orcid":"","institution":"University of New England","correspondingAuthor":false,"prefix":"","firstName":"Anna-Marie","middleName":"","lastName":"Babey","suffix":""},{"id":627767594,"identity":"6a008a71-02a5-4c5c-854f-04d1f2771d1a","order_by":3,"name":"Willmann Liang","email":"","orcid":"","institution":"University of Hong Kong","correspondingAuthor":false,"prefix":"","firstName":"Willmann","middleName":"","lastName":"Liang","suffix":""},{"id":627767595,"identity":"fce2833d-e1ab-4bc5-a805-6cc1459422f8","order_by":4,"name":"Martin Hawes","email":"","orcid":"","institution":"University of Surrey","correspondingAuthor":false,"prefix":"","firstName":"Martin","middleName":"","lastName":"Hawes","suffix":""},{"id":627767597,"identity":"f93c3cc5-e482-4f95-98fa-b754144d28ac","order_by":5,"name":"Graeme Sills","email":"","orcid":"","institution":"University of Glasgow","correspondingAuthor":false,"prefix":"","firstName":"Graeme","middleName":"","lastName":"Sills","suffix":""},{"id":627767599,"identity":"8c7327d3-5d67-4931-ab33-223f0ea9332b","order_by":6,"name":"Olusola Olafuyi","email":"","orcid":"","institution":"University of Nottingham","correspondingAuthor":false,"prefix":"","firstName":"Olusola","middleName":"","lastName":"Olafuyi","suffix":""},{"id":627767602,"identity":"eecbedd2-697e-4517-9063-4436b60126ab","order_by":7,"name":"Carolina Restini","email":"","orcid":"","institution":"Michigan State University","correspondingAuthor":false,"prefix":"","firstName":"Carolina","middleName":"","lastName":"Restini","suffix":""},{"id":627767603,"identity":"7acd0b90-2b8e-4feb-babb-9c9e9ec5e1e1","order_by":8,"name":"Marina Junqueira Santiago","email":"","orcid":"","institution":"Macquarie University","correspondingAuthor":false,"prefix":"","firstName":"Marina","middleName":"Junqueira","lastName":"Santiago","suffix":""},{"id":627767604,"identity":"513cebc3-90b1-4139-a34b-ab5f7aff1a7e","order_by":9,"name":"Jennelle Durnett Richardson","email":"","orcid":"","institution":"Indiana University","correspondingAuthor":false,"prefix":"","firstName":"Jennelle","middleName":"Durnett","lastName":"Richardson","suffix":""},{"id":627767605,"identity":"f338d201-6390-484a-aba2-b6685d2f9cdc","order_by":10,"name":"Anneke van Houwelingen","email":"","orcid":"","institution":"Utrecht University","correspondingAuthor":false,"prefix":"","firstName":"Anneke","middleName":"van","lastName":"Houwelingen","suffix":""},{"id":627767606,"identity":"0319112f-b1fa-4cec-9dbf-485bb35c25b1","order_by":11,"name":"Elvan Djouma","email":"","orcid":"","institution":"La Trobe University","correspondingAuthor":false,"prefix":"","firstName":"Elvan","middleName":"","lastName":"Djouma","suffix":""},{"id":627767607,"identity":"9e132a0a-585e-4aaa-a6fe-77dcbff9689a","order_by":12,"name":"Lynette Fernandes","email":"","orcid":"","institution":"University of Western Australia","correspondingAuthor":false,"prefix":"","firstName":"Lynette","middleName":"","lastName":"Fernandes","suffix":""},{"id":627767608,"identity":"0cc870e7-e386-4087-a6bd-e68f2a7f7106","order_by":13,"name":"Lindsay Cormier","email":"","orcid":"","institution":"University of Kentucky","correspondingAuthor":false,"prefix":"","firstName":"Lindsay","middleName":"","lastName":"Cormier","suffix":""},{"id":627767609,"identity":"995514f4-e9f1-44d3-90ee-435807e57350","order_by":14,"name":"Fabiana A Caetano Crowley","email":"","orcid":"","institution":"Western University","correspondingAuthor":false,"prefix":"","firstName":"Fabiana","middleName":"A Caetano","lastName":"Crowley","suffix":""},{"id":627767610,"identity":"2e862b74-3919-4ed1-b0bf-c25053eef4d1","order_by":15,"name":"Paul J. White","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA6UlEQVRIiWNgGAWjYHACxgMMDAd4GBiYD4C5BkAsQUgPWAsPA1sCaVoYeBh4DIjTYs5+xuAAY9sdGXv2nq8bPu6wszdnYD54mwePFsueHJCWZzw8PGe33Zx5JjlxZwNbsjU+LQYHQFq2Hebhkcjddpu3jTnB4ACPmTReLeffQLXIv3kG1FJvb3CA/xt+LTfgtvCwAbUcZtxwgIcNrxbLGc8KDiT+A2o5k2Z2c2bb8cQNh9mMLefg0WLOn7zxwYczh+3Z2w8/u/Gxrdre4Hjzwxtv8DkMRCSgCDHjUQ7XMgpGwSgYBaMALwAAbp1RLynX23gAAAAASUVORK5CYII=","orcid":"","institution":"Monash University","correspondingAuthor":true,"prefix":"","firstName":"Paul","middleName":"J.","lastName":"White","suffix":""},{"id":627767611,"identity":"b024e275-6ce8-4aa0-aeeb-73776aa92110","order_by":16,"name":"Will Parsons","email":"","orcid":"","institution":"Monash University","correspondingAuthor":false,"prefix":"","firstName":"Will","middleName":"","lastName":"Parsons","suffix":""}],"badges":[],"createdAt":"2026-03-10 01:09:20","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9077666/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9077666/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":107694151,"identity":"c0640cc5-ef2f-4c87-99f8-1e699c4b7a1c","added_by":"auto","created_at":"2026-04-24 06:41:09","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1271577,"visible":true,"origin":"","legend":"","description":"","filename":"RevisedManuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9077666/v1_covered_a49e255a-59a3-444c-82a5-da19ea00c5cd.pdf"},{"id":107693987,"identity":"a3f4f1df-1b56-4968-9db2-e378bd6fb52b","added_by":"auto","created_at":"2026-04-24 06:40:19","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":25354,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementaryfile1Table1DemographiccharacteristicsofrespondentsinthepilotsurveyN70.docx","url":"https://assets-eu.researchsquare.com/files/rs-9077666/v1/413970dae6ae835c95a60f8b.docx"},{"id":107694055,"identity":"0a91a5b4-7b32-47a0-9c41-f9c4c82f7ae1","added_by":"auto","created_at":"2026-04-24 06:40:32","extension":"docx","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":23141,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementaryfile2Table2NumberofopenendedresponsesbyPCIquestion.docx","url":"https://assets-eu.researchsquare.com/files/rs-9077666/v1/515089212ced26dc2cb44d68.docx"},{"id":107694074,"identity":"07e9c376-44d4-48d9-a5b3-6c3d6e0191d5","added_by":"auto","created_at":"2026-04-24 06:40:45","extension":"docx","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":23269,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementaryfile3Table3Cognitiveinterviewprotocol.docx","url":"https://assets-eu.researchsquare.com/files/rs-9077666/v1/c623d6cb954c2dafa2568c3b.docx"},{"id":107693986,"identity":"acaf246e-5c01-407a-b184-74f2f3b1fe03","added_by":"auto","created_at":"2026-04-24 06:40:19","extension":"docx","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":28510,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryFile4Interviewguidethinkaloudandprobingapproach.docx","url":"https://assets-eu.researchsquare.com/files/rs-9077666/v1/b4e06a28b9b292d0c9d10b96.docx"},{"id":107693993,"identity":"e8f70293-e6b6-4f6c-8246-121694258f5d","added_by":"auto","created_at":"2026-04-24 06:40:20","extension":"docx","order_by":6,"title":"","display":"","copyAsset":false,"role":"supplement","size":224925,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementaryfile5Figure1ReasoningdistributionacrosspharmacokineticsPCIitems.docx","url":"https://assets-eu.researchsquare.com/files/rs-9077666/v1/eecffd19c779d5c6b4aa12a7.docx"},{"id":107693997,"identity":"1da8d25a-5a57-49b4-86b7-4db89b0d893c","added_by":"auto","created_at":"2026-04-24 06:40:27","extension":"docx","order_by":7,"title":"","display":"","copyAsset":false,"role":"supplement","size":211847,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementaryfile6Figure2ReasoningdistributionacrosspharmacodynamicsPCIitems.docx","url":"https://assets-eu.researchsquare.com/files/rs-9077666/v1/f8b88f64b525eca1f197a259.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Inside the learners’ minds: Exploring response-process validity and diagnosing misconceptions in the Pharmacology Concept Inventory","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"pharmacology concept inventory, response process validity, misconceptions, cognitive interview","lastPublishedDoi":"10.21203/rs.3.rs-9077666/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9077666/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eEvaluating pharmacology teaching and learning methods requires assessment tools with strong evidence of validity. The Pharmacology Concept Inventory (PCI) was developed to measure concept-based learning in pharmacology. To achieve this, evidence from learners\u0026rsquo; perspectives must be collected alongside expert-guided refinement of the tool. This study aimed to gather evidence for the response process validity (RPV) of the PCI, identify new misconceptions, and highlight areas for improvement. We adopted a qualitative approach, drawing on multiple sources: analyzing students\u0026rsquo; open-ended responses (OERs), conducting inductive thematic analysis of cognitive interview data, and involving experts in item review. Of the 70 students invited, about two-thirds submitted an OER, 16 participated in think-aloud cognitive interviews, and 5 experts contributed to item refinement. The OERs revealed common patterns of understanding, misconceptions, and uncertainty. The thematic analysis of cognitive interviews uncovered three main themes: interpretation and understanding of items; pharmacological reasoning and evidence use; and validation of conceptual understanding and distractor function, including the emergence of new misconceptions. These findings helped develop a cognitive diagnostic framework that provides evidence for RPV and allows systematic detection of emerging misconceptions in the PCI. The study clarified how learners interpret PCI items and how scientific explanations or misconceptions influence their decision-making. Correct, scientifically reasoned responses were linked with proper understanding, while distractor analysis highlighted recurring misconceptions and new alternative ideas. Overall, these results support strategies for collecting RPV evidence, identifying misconceptions, and refining assessment items to enhance concept-based pharmacology education.\u003c/p\u003e","manuscriptTitle":"Inside the learners’ minds: Exploring response-process validity and diagnosing misconceptions in the Pharmacology Concept Inventory","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-24 06:38:25","doi":"10.21203/rs.3.rs-9077666/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewerAgreed","content":"276472040890054141612899973037581158601","date":"2026-04-30T06:09:03+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-04-17T10:08:01+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-04-13T08:48:01+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2026-03-19T10:40:02+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-03-18T17:23:57+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Education","date":"2026-03-18T00:44:36+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"09cec665-0ee4-43a1-bd3b-991c39462644","owner":[],"postedDate":"April 24th, 2026","published":true,"recentEditorialEvents":[{"type":"reviewerAgreed","content":"276472040890054141612899973037581158601","date":"2026-04-30T06:09:03+00:00","index":70,"fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-04-24T06:38:26+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-24 06:38:25","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9077666","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9077666","identity":"rs-9077666","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}