Classification of Brain Tumours in MR Images using Deep Spatiospatial Models

Classification of Brain Tumours in MR Images using Deep Spatiospatial Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Classification of Brain Tumours in MR Images using Deep Spatiospatial Models Soumick Chatterjee, Faraz Ahmed Nizamani, Andreas Nürnberger, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-570429/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 27 Jan, 2022 Read the published version in Scientific Reports → Version 1 posted 9 You are reading this latest preprint version Abstract A brain tumour is a mass or cluster of abnormal cells in the brain, which has the possibility of becoming life-threatening because of its ability to invade neighbouring tissues and also form metastases. An accurate diagnosis is essential for successful treatment planning and magnetic resonance imaging is the principal imaging modality for diagnostic of brain tumours and their extent. Deep Learning methods in computer vision applications have shown significant improvement in recent years, most of which can be credited to the fact that a sizeable amount of data is available to train models on, and the improvements in the model architectures yielding better approximations in a supervised setting. Classifying tumours using such deep learning methods has made significant progress with the availability of open datasets with reliable annotations. Typically those methods are either 3D models, which use 3D volumetric MRIs or even 2D models considering each slice separately. However, by treating one spatial dimension separately or by considering the slices as a sequence of images over time, spatiotemporal models can be employed as "spatiospatial" models for this task. These models have the capabilities of learning specific spatial and temporal relationship, while reducing computational costs. This paper uses two spatiotemporal models, ResNet (2+1)D and ResNet Mixed Convolution, to classify different types of brain tumours. It was observed that both these models performed superior to the pure 3D convolutional model, ResNet18. Furthermore, it was also observed that pre-training the models on a different, even unrelated dataset before training them for the task of tumour classification improves the performance. Finally, Pre-trained ResNet Mixed Convolution was observed to be the best model in these experiments, achieving a macro F1-score of 0.9345 and a test accuracy of 96.98%, while at the same time being the model with the least computational cost. Scientific Communication Cancer Biology brain tumour abnormal cells Deep Spatiospatial Models Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 27 Jan, 2022 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Major revision 06 Sep, 2021 Reviews received at journal 16 Aug, 2021 Reviewers agreed at journal 11 Aug, 2021 Reviewers agreed at journal 27 Jul, 2021 Reviewers invited by journal 19 Jul, 2021 Editor assigned by journal 19 Jul, 2021 Editor invited by journal 04 Jun, 2021 Submission checks completed at journal 02 Jun, 2021 First submitted to journal 28 May, 2021 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-570429","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":30739682,"identity":"2a1f349d-bb47-4325-97e3-2cecda63c9cb","order_by":0,"name":"Soumick Chatterjee","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABGklEQVRIiWNgGAWjYLCCBCBkkICwmYHoAIR5gLAWxoYDYC1sCYS1MCBpAQIeA7xa5Nt7TDc8qElj4J/d/Pzxh4p77PzsPR8f8/w6zMB3vAGrFoMzZ8xuJBzLYZC4c8yw4cCZYmbJnrObjXn7DjNInsFujYFEjtmNxIYKICPBsOFgWwKzwY3cbZIzew4zGNxIwO6wGXAt6R8bDv5LYLa//+YZRMv9B9g9cwOsJQdkHdCWBqAtEjxsEh9+gGzBrsPgzLEyoF/SeCRu5BTOOHMsgVniTJqxwceGdB7JMzgc1t687eaPmmQ5/hnpGz5U1CQk87cffvgg4Y+1HN9x7N6HAR4YIxlMMrYhRAgCOwj1h2gNo2AUjIJRMPwBAFbqaey4MhfuAAAAAElFTkSuQmCC","orcid":"","institution":"Otto von Guericke University Magdeburg","correspondingAuthor":true,"prefix":"","firstName":"Soumick","middleName":"","lastName":"Chatterjee","suffix":""},{"id":30739684,"identity":"f181435d-a45e-48d2-9d22-3aed004e7db4","order_by":1,"name":"Faraz Ahmed Nizamani","email":"","orcid":"","institution":"Otto von Guericke University Magdeburg","correspondingAuthor":false,"prefix":"","firstName":"Faraz","middleName":"Ahmed","lastName":"Nizamani","suffix":""},{"id":30739686,"identity":"b916fba2-b04b-4d6b-8f6a-8763dce1d6af","order_by":2,"name":"Andreas Nürnberger","email":"","orcid":"","institution":"Otto von Guericke University Magdeburg","correspondingAuthor":false,"prefix":"","firstName":"Andreas","middleName":"","lastName":"Nürnberger","suffix":""},{"id":30739687,"identity":"90f13a4c-9617-4914-9c36-8e8b9b1dab6c","order_by":3,"name":"Oliver Speck","email":"","orcid":"","institution":"Otto von Guericke University Magdeburg","correspondingAuthor":false,"prefix":"","firstName":"Oliver","middleName":"","lastName":"Speck","suffix":""}],"badges":[],"createdAt":"2021-05-28 18:29:07","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-570429/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-570429/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-022-05572-6","type":"published","date":"2022-01-27T13:35:49+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":9987438,"identity":"f0439a99-6be5-4baa-af66-83dd7eb5edd0","added_by":"auto","created_at":"2021-06-04 14:40:06","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":83917,"visible":true,"origin":"","legend":"Please see the Manuscript PDF file for the complete figure caption","description":"","filename":"1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-570429/v1/13949cf408017565f8f886c3.jpg"},{"id":9987385,"identity":"88893f89-4260-4d3f-ae7c-84b732e05bad","added_by":"auto","created_at":"2021-06-04 14:37:06","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":86230,"visible":true,"origin":"","legend":"Please see the Manuscript PDF file for the complete figure caption","description":"","filename":"2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-570429/v1/8608a7c3d4da4f713cc2dd26.jpg"},{"id":9987387,"identity":"bd85dd41-8dd2-4621-bce7-29ee0c9091ce","added_by":"auto","created_at":"2021-06-04 14:37:06","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":388401,"visible":true,"origin":"","legend":"Please see the Manuscript PDF file for the complete figure caption","description":"","filename":"3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-570429/v1/8258746ecb908a763c065a52.jpg"},{"id":9987188,"identity":"a6c83db5-1b09-4129-a026-e4129581be3b","added_by":"auto","created_at":"2021-06-04 14:34:06","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":143233,"visible":true,"origin":"","legend":"Please see the Manuscript PDF file for the complete figure caption","description":"","filename":"4.jpg","url":"https://assets-eu.researchsquare.com/files/rs-570429/v1/f52c3ee90b9df5c992d74a65.jpg"},{"id":9987388,"identity":"c11036e3-78d9-4e50-929c-16ef81140ef7","added_by":"auto","created_at":"2021-06-04 14:37:06","extension":"jpg","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":144985,"visible":true,"origin":"","legend":"Please see the Manuscript PDF file for the complete figure caption","description":"","filename":"5.jpg","url":"https://assets-eu.researchsquare.com/files/rs-570429/v1/02e5ffb262630cc9a4a97164.jpg"},{"id":9987191,"identity":"9f520bf9-5cfd-473c-9826-419721f9f4d0","added_by":"auto","created_at":"2021-06-04 14:34:06","extension":"jpg","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":147795,"visible":true,"origin":"","legend":"Please see the Manuscript PDF file for the complete figure caption","description":"","filename":"6.jpg","url":"https://assets-eu.researchsquare.com/files/rs-570429/v1/6e1c541da45334020300b51e.jpg"},{"id":9987194,"identity":"ebf98b32-9699-46bf-ab51-fe206e9c5b75","added_by":"auto","created_at":"2021-06-04 14:34:06","extension":"jpg","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":284371,"visible":true,"origin":"","legend":"Please see the Manuscript PDF file for the complete figure caption","description":"","filename":"7.jpg","url":"https://assets-eu.researchsquare.com/files/rs-570429/v1/fdee79a177ae72b7c76a66c5.jpg"},{"id":17697477,"identity":"73338407-4caa-461d-878f-8ff52116a100","added_by":"auto","created_at":"2022-01-27 13:35:55","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1068423,"visible":true,"origin":"","legend":"","description":"","filename":"TumourClassifySpatiospatial.pdf","url":"https://assets-eu.researchsquare.com/files/rs-570429/v1_covered.pdf"},{"id":13640056,"identity":"479f4bd7-6e0c-4234-81ee-c9e2b745cf5a","added_by":"auto","created_at":"2021-09-17 08:58:22","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1063150,"visible":true,"origin":"","legend":"","description":"","filename":"TumourClassifySpatiospatial.pdf","url":"https://assets-eu.researchsquare.com/files/rs-570429/v1_covered.pdf"},{"id":9987439,"identity":"77c65226-341c-4bec-af83-6199ff275b09","added_by":"auto","created_at":"2021-06-04 14:40:11","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1059524,"visible":true,"origin":"","legend":"","description":"","filename":"TumourClassifySpatiospatial.pdf","url":"https://assets-eu.researchsquare.com/files/rs-570429/v1_covered.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Classification of Brain Tumours in MR Images using Deep Spatiospatial Models","fulltext":[{"header":"Full Text","content":"This preprint is available for \u003ca href='/article/rs-570429/latest.pdf' target='_blank'\u003edownload as a PDF\u003c/a\u003e."}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"brain tumour, abnormal cells, Deep Spatiospatial Models","lastPublishedDoi":"10.21203/rs.3.rs-570429/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-570429/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"A brain tumour is a mass or cluster of abnormal cells in the brain, which has the possibility of becoming life-threatening because of its ability to invade neighbouring tissues and also form metastases. An accurate diagnosis is essential for successful treatment planning and magnetic resonance imaging is the principal imaging modality for diagnostic of brain tumours and their extent. Deep Learning methods in computer vision applications have shown significant improvement in recent years, most of which can be credited to the fact that a sizeable amount of data is available to train models on, and the improvements in the model architectures yielding better approximations in a supervised setting. Classifying tumours using such deep learning methods has made significant progress with the availability of open datasets with reliable annotations. Typically those methods are either 3D models, which use 3D volumetric MRIs or even 2D models considering each slice separately. However, by treating one spatial dimension separately or by considering the slices as a sequence of images over time, spatiotemporal models can be employed as \"spatiospatial\" models for this task. These models have the capabilities of learning specific spatial and temporal relationship, while reducing computational costs. This paper uses two spatiotemporal models, ResNet (2+1)D and ResNet Mixed Convolution, to classify different types of brain tumours. It was observed that both these models performed superior to the pure 3D convolutional model, ResNet18. Furthermore, it was also observed that pre-training the models on a different, even unrelated dataset before training them for the task of tumour classification improves the performance. Finally, Pre-trained ResNet Mixed Convolution was observed to be the best model in these experiments, achieving a macro F1-score of 0.9345 and a test accuracy of 96.98%, while at the same time being the model with the least computational cost.","manuscriptTitle":"Classification of Brain Tumours in MR Images using Deep Spatiospatial Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2021-06-04 14:34:04","doi":"10.21203/rs.3.rs-570429/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Major revision","date":"2021-09-06T08:47:43+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2021-08-16T09:09:51+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"d15a06f9-8890-49ee-bd3b-742aae05401e","date":"2021-08-11T14:30:30+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"048f230b-991b-4879-9063-1544547acf61","date":"2021-07-27T08:32:55+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2021-07-19T20:52:08+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2021-07-19T20:47:23+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2021-06-04T14:26:20+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2021-06-02T16:58:34+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2021-05-28T18:27:15+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"44f212d2-b255-4318-a234-a10bad0cf56a","owner":[],"postedDate":"June 4th, 2021","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":4799058,"name":"Scientific Communication"},{"id":4799059,"name":"Cancer Biology"}],"tags":[],"updatedAt":"2022-01-27T13:35:49+00:00","versionOfRecord":{"articleIdentity":"rs-570429","link":"https://doi.org/10.1038/s41598-022-05572-6","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2022-01-27 13:35:49","publishedOnDateReadable":"January 27th, 2022"},"versionCreatedAt":"2021-06-04 14:34:04","video":"","vorDoi":"10.1038/s41598-022-05572-6","vorDoiUrl":"https://doi.org/10.1038/s41598-022-05572-6","workflowStages":[]},"version":"v1","identity":"rs-570429","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-570429","identity":"rs-570429","version":["v1"]},"buildId":"_2-kVJe1T_tPrBINL-cwx","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}