Attention Re-Alignment in Multimodal Large Language Models via Intermediate-Layer Guidance

Attention Re-Alignment in Multimodal Large Language Models via Intermediate-Layer Guidance | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Attention Re-Alignment in Multimodal Large Language Models via Intermediate-Layer Guidance Yanming Chen, Pandong Wang, Guofeng Qin, Wei Wu, Ming Chen, Yongtao Hao This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7956158/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 23 Mar, 2026 Read the published version in Scientific Reports → Version 1 posted 12 You are reading this latest preprint version Abstract Multimodal large language models (MLLMs) have achieved impressive performance in understanding and describing visual content, setting new state-of-the-art results on a variety of visual question answering (VQA) benchmarks. However, during decoding, these models often fail to attend to fine-grained visual details in the input image. Our analysis of intermediate attention layers reveals that MLLMs are not inherently incapable of perceiving target objects; rather, attention to visual details becomes diluted in deeper layers due to the dominance of language priors. To address this limitation, we propose a plug-and-play Attention Re-Alignment module (ARA) that enhances suppressed visual grounding. ARA conducts a layer-wise analysis of the relative attention distribution of image-centric attention heads. It incorporates a confidence-aware layer selection mechanism based on attention peak and entropy, enabling the dynamic aggregation of attention maps from the most informative layers. These aggregated maps are subsequently leveraged to guide the generation of semantic masks, enabling the model to emphasize salient visual regions while suppressing irrelevant or noisy content. ARA can be seamlessly integrated into existing MLLMs and demonstrates consistent improvements across multiple VQA benchmarks, validating its effectiveness in enhancing visual detail sensitivity. Physical sciences/Mathematics and computing Biological sciences/Neuroscience Biological sciences/Psychology Social science/Psychology Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 23 Mar, 2026 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Revision requested 21 Nov, 2025 Reviewers agreed at journal 19 Nov, 2025 Reviewers agreed at journal 19 Nov, 2025 Reviews received at journal 18 Nov, 2025 Reviews received at journal 05 Nov, 2025 Reviewers agreed at journal 05 Nov, 2025 Reviewers agreed at journal 04 Nov, 2025 Reviewers invited by journal 04 Nov, 2025 Editor assigned by journal 04 Nov, 2025 Editor invited by journal 04 Nov, 2025 Submission checks completed at journal 03 Nov, 2025 First submitted to journal 03 Nov, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7956158","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":540472241,"identity":"849c3a0c-83a0-452d-a726-40148966865f","order_by":0,"name":"Yanming Chen","email":"","orcid":"","institution":"Tongji University","correspondingAuthor":false,"prefix":"","firstName":"Yanming","middleName":"","lastName":"Chen","suffix":""},{"id":540472242,"identity":"5fdedc86-ab5a-45e6-a270-0d42328e8382","order_by":1,"name":"Pandong Wang","email":"","orcid":"","institution":"Tongji University","correspondingAuthor":false,"prefix":"","firstName":"Pandong","middleName":"","lastName":"Wang","suffix":""},{"id":540472243,"identity":"94cae2ef-ddf3-4fb8-b365-c82d8bd04264","order_by":2,"name":"Guofeng Qin","email":"","orcid":"","institution":"Tongji University","correspondingAuthor":false,"prefix":"","firstName":"Guofeng","middleName":"","lastName":"Qin","suffix":""},{"id":540472244,"identity":"919d0a7d-57d3-48b0-ba1c-70170ec19ec4","order_by":3,"name":"Wei Wu","email":"","orcid":"","institution":"Tongji University","correspondingAuthor":false,"prefix":"","firstName":"Wei","middleName":"","lastName":"Wu","suffix":""},{"id":540472245,"identity":"4cacf072-bd21-49b4-86de-1e2dcf87dfd4","order_by":4,"name":"Ming Chen","email":"","orcid":"","institution":"Fuyao University of Science and Technology","correspondingAuthor":false,"prefix":"","firstName":"Ming","middleName":"","lastName":"Chen","suffix":""},{"id":540472246,"identity":"85fa6538-e189-4e9c-9926-316c16f88119","order_by":5,"name":"Yongtao Hao","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA2ElEQVRIiWNgGAWjYBACAzBZASYTDjAwMBOp5cAZA1K1HGwzgPGJ0GLOfviY9Md5f+TM+Rc8PMBQYZ3YwH72AF4tlj1paRIHtxkYW854AHTYmfTEBp68BPwOO5BjBtKSuOHGgYQDjG2HExskeAzwazn/BqhlDkzLP2K03ADZ0gDUcr4BqKWBKC3Pki3OHDM2NrgBDOSEY+nGbTw5hByWfPBGRY2cnMH5M8kfPtRYy/azn8GvBQhYJMCURE4CQwKQZiOkHgiYP4Ap/uMHiFA8CkbBKBgFIxEAAK7WT3FMGnKyAAAAAElFTkSuQmCC","orcid":"","institution":"Tongji University","correspondingAuthor":true,"prefix":"","firstName":"Yongtao","middleName":"","lastName":"Hao","suffix":""}],"badges":[],"createdAt":"2025-10-27 11:43:52","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7956158/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7956158/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-026-44935-1","type":"published","date":"2026-03-23T16:10:22+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":95958728,"identity":"05807723-3940-4e14-a45f-50fc95f0bc82","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":8011,"visible":true,"origin":"","legend":"","description":"","filename":"c2c0411c48c749aba6155e35fd786e9b.json","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/647165cef05795d828cf9d1e.json"},{"id":95958729,"identity":"9d305bde-4740-4afb-bce2-3f0762ffa388","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":103955,"visible":true,"origin":"","legend":"","description":"","filename":"Comment.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/5273c7c0c65b09578ae152b1.pdf"},{"id":96245373,"identity":"a248e74e-5c03-4c4e-81e0-26dd98f1cf69","added_by":"auto","created_at":"2025-11-19 07:20:29","extension":"jpg","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":347009,"visible":true,"origin":"","legend":"","description":"","filename":"image1a.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/a6760b08b0b938b96ced4920.jpg"},{"id":95958734,"identity":"29c1ef69-de2e-406a-8588-40337546ffb3","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"jpg","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":342687,"visible":true,"origin":"","legend":"","description":"","filename":"image1b.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/e8b3ea0472758d37664f9fc3.jpg"},{"id":95958736,"identity":"824cc06e-f1b0-43d1-bf3a-67ef355ae915","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"jpg","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":148441,"visible":true,"origin":"","legend":"","description":"","filename":"image2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/d2ae86a779936f0f2e8b5d96.jpg"},{"id":95958730,"identity":"02bba2eb-69cc-47cb-8615-c2acbdbcca43","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"jpg","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":81713,"visible":true,"origin":"","legend":"","description":"","filename":"image3a.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/fe10276d73093a584121c780.jpg"},{"id":95958732,"identity":"85219c56-3c60-4629-80ee-90ae627d7cab","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"jpg","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":505059,"visible":true,"origin":"","legend":"","description":"","filename":"image3b.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/a04e6a0323f7f308b3aeccf7.jpg"},{"id":95958735,"identity":"62a34fd0-8644-4d29-8717-16dce241a7dd","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"jpg","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":292327,"visible":true,"origin":"","legend":"","description":"","filename":"image4.jpg","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/6c54f75c37b741b0f11b77a6.jpg"},{"id":96244839,"identity":"a8bf67fb-078a-49d7-a7a2-29563fc826f7","added_by":"auto","created_at":"2025-11-19 07:19:23","extension":"ldf","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":49107,"visible":true,"origin":"","legend":"","description":"","filename":"jabbrvltwaall.ldf","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/ce4eb63b54e947712b125ca9.ldf"},{"id":95958742,"identity":"a6ee422a-497f-457b-b4b0-03c2670e35e1","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"ldf","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":268657,"visible":true,"origin":"","legend":"","description":"","filename":"jabbrvltwaen.ldf","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/6d56047f5651f15017165c31.ldf"},{"id":95958740,"identity":"5d8c5368-c787-430e-ae03-0b92d7d7c3a3","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"sty","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":15480,"visible":true,"origin":"","legend":"","description":"","filename":"jabbrv.sty","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/f9718358dc52b85a3c9777e9.sty"},{"id":95958743,"identity":"bbe28f4b-2066-4799-8490-69de260c37e4","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"pdf","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":2135157,"visible":true,"origin":"","legend":"","description":"","filename":"jstmjjwwszxxgyypxprhdrhnpfbdmngm.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/48a30c91f5949c5c5cbdfb0b.pdf"},{"id":95958731,"identity":"753c6c57-f565-47ab-9db2-e957a6fa51bb","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"bbl","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":13977,"visible":true,"origin":"","legend":"","description":"","filename":"main.bbl","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/da44d15c55d29da7c9edeead.bbl"},{"id":95958738,"identity":"1a47a4b9-fd7f-43cf-b1f2-8cbc88a653a3","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"bst","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":36153,"visible":true,"origin":"","legend":"","description":"","filename":"naturemagdoi.bst","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/c3804f150ca62fe719cf440d.bst"},{"id":96245581,"identity":"f7a8a368-f38b-448b-a8da-debef5eec8e8","added_by":"auto","created_at":"2025-11-19 07:21:04","extension":"cls","order_by":14,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":5824,"visible":true,"origin":"","legend":"","description":"","filename":"wlscirep.cls","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/9e49550f2562b1b352963fd7.cls"},{"id":95958737,"identity":"5ce1056e-92ab-42a9-a694-82689e01cd47","added_by":"auto","created_at":"2025-11-14 23:46:15","extension":"xml","order_by":15,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":64747,"visible":true,"origin":"","legend":"","description":"","filename":"c2c0411c48c749aba6155e35fd786e9b1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1/606092d5db38341ea2d23436.xml"},{"id":105755720,"identity":"3999a9f3-380d-4a87-bb2a-380b27f8960f","added_by":"auto","created_at":"2026-03-30 16:29:54","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":862344,"visible":true,"origin":"","legend":"","description":"","filename":"jstmjjwwszxxgyypxprhdrhnpfbdmngm.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7956158/v1_covered_f49f2cba-a65a-4bcf-98ef-ba24df0f2171.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Attention Re-Alignment in Multimodal Large Language Models via Intermediate-Layer Guidance","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-7956158/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7956158/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Multimodal large language models (MLLMs) have achieved impressive performance in understanding and describing visual content, setting new state-of-the-art results on a variety of visual question answering (VQA) benchmarks. However, during decoding, these models often fail to attend to fine-grained visual details in the input image. Our analysis of intermediate attention layers reveals that MLLMs are not inherently incapable of perceiving target objects; rather, attention to visual details becomes diluted in deeper layers due to the dominance of language priors. To address this limitation, we propose a plug-and-play Attention Re-Alignment module (ARA) that enhances suppressed visual grounding. ARA conducts a layer-wise analysis of the relative attention distribution of image-centric attention heads. It incorporates a confidence-aware layer selection mechanism based on attention peak and entropy, enabling the dynamic aggregation of attention maps from the most informative layers. These aggregated maps are subsequently leveraged to guide the generation of semantic masks, enabling the model to emphasize salient visual regions while suppressing irrelevant or noisy content. ARA can be seamlessly integrated into existing MLLMs and demonstrates consistent improvements across multiple VQA benchmarks, validating its effectiveness in enhancing visual detail sensitivity.","manuscriptTitle":"Attention Re-Alignment in Multimodal Large Language Models via Intermediate-Layer Guidance","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-11-14 23:46:10","doi":"10.21203/rs.3.rs-7956158/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-11-21T12:23:23+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"246581129239710371402515609799511267112","date":"2025-11-19T07:45:53+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"228261608846813804384052014495023881178","date":"2025-11-19T05:04:53+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-18T18:29:45+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-11-05T14:27:24+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"51536078747371145344400010416345584357","date":"2025-11-05T14:21:47+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"316862409021998031703650125782820702951","date":"2025-11-05T03:28:01+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-11-05T03:26:38+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-11-05T03:20:51+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-11-04T17:18:34+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-11-03T17:40:11+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2025-11-03T08:34:44+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"8b2e6ea2-179c-4baf-b723-5a84812d4e32","owner":[],"postedDate":"November 14th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":57493339,"name":"Physical sciences/Mathematics and computing"},{"id":57493340,"name":"Biological sciences/Neuroscience"},{"id":57493341,"name":"Biological sciences/Psychology"},{"id":57493342,"name":"Social science/Psychology"}],"tags":[],"updatedAt":"2026-03-30T16:25:18+00:00","versionOfRecord":{"articleIdentity":"rs-7956158","link":"https://doi.org/10.1038/s41598-026-44935-1","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2026-03-23 16:10:22","publishedOnDateReadable":"March 23rd, 2026"},"versionCreatedAt":"2025-11-14 23:46:10","video":"","vorDoi":"10.1038/s41598-026-44935-1","vorDoiUrl":"https://doi.org/10.1038/s41598-026-44935-1","workflowStages":[]},"version":"v1","identity":"rs-7956158","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7956158","identity":"rs-7956158","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}