{"paper_id":"49d80d5c-025d-47e7-b282-8fcfe50d00a3","body_text":"Exploring Semanticity-Based Clustering of Text Using Transformer Models: Advancing AI Applications in Education and Beyond | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Exploring Semanticity-Based Clustering of Text Using Transformer Models: Advancing AI Applications in Education and Beyond Shreya Suresh, Jeganathan L, Janaki Meena M, Srinivasa Rao Ummity, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6672315/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The study explores semantic-based clustering using transformer models to overcome the limitations of traditional text clustering approaches. While conventional methods rely on word frequency, this research leverages BERT and SciBERT's contextual understanding capabilities for more nuanced text organization. The methodology combines transformer-based semantic embeddings with various pooling strategies and clustering algorithms, comparing their performance against TF-IDF baselines. Experiments extended across five diverse domains: news, research papers, e-commerce products, movies, and job postings. It was observed that transformer-based embeddings with CLS pooling consistently outperformed traditional methods, producing more coherent clusters across all domains. SciBERT proved to be particularly useful for scientific text. These findings show possible applications in personalized learning systems, content organization, and recommender systems where semantic interpretation is critical. The research provides a framework to develop text clustering solutions better suited to capture contextual linkages and semantic intricacies in complex document collections. Biological sciences/Computational biology and bioinformatics/Machine learning Physical sciences/Mathematics and computing Physical sciences/Mathematics and computing/Computer science Semantic Clustering Transformer Models BERT and SciBERT Educational Applications and Text Embedding and Pooling Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {\"props\":{\"pageProps\":{\"initialData\":{\"identity\":\"rs-6672315\",\"acceptedTermsAndConditions\":true,\"allowDirectSubmit\":true,\"archivedVersions\":[],\"articleType\":\"Article\",\"associatedPublications\":[],\"authors\":[{\"id\":489255936,\"identity\":\"3ea65886-0b2a-4ece-9dfb-33a40289339f\",\"order_by\":0,\"name\":\"Shreya Suresh\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Vellore Institute of Technology University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Shreya\",\"middleName\":\"\",\"lastName\":\"Suresh\",\"suffix\":\"\"},{\"id\":489255938,\"identity\":\"c760e2e1-b9e3-49c5-bf1c-ecc21e52d0a6\",\"order_by\":1,\"name\":\"Jeganathan L\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Vellore Institute of Technology University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Jeganathan\",\"middleName\":\"\",\"lastName\":\"L\",\"suffix\":\"\"},{\"id\":489255939,\"identity\":\"2c246642-d7b7-4cc7-97ce-9c82124a65d9\",\"order_by\":2,\"name\":\"Janaki Meena M\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Vellore Institute of Technology University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Janaki\",\"middleName\":\"Meena\",\"lastName\":\"M\",\"suffix\":\"\"},{\"id\":489255940,\"identity\":\"b8e0a1a6-2e12-4e08-93d8-c0457e6c1e95\",\"order_by\":3,\"name\":\"Srinivasa Rao Ummity\",\"email\":\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABFElEQVRIie3SMUvDQBTA8RcCnc5mPZf0K1wppIuSr5IjUJcOdRFBCemSbM5v6Ie58KAuwVGULAXByeGK4BTEpKEI8aKrw/2XhLz8uBwXAJvtX+ak3dUFUMdHOxDdnfr5fjtfp6pHXAFC/EKaZfqDEYfjMobEI+Uv+wzCeX5SELtNJvOctjdsVYOXK4dWBlLJdVpkIDc0johtabopF4uKNR/GywgIh0nEXSYKHCkHYRk8Y7uXJwBiwyTsyGcSovcWXLZk8gdxsCFqn7kS+TIA3RAxQE4bguUDlx25oxj5a8y1mLFpKVMTGVcXO319dRZyr5zp6CM5Ry8udFT7vn9P9G4gh5z2IPqx7x/DVD08stlsNht8AZtYakXOdbwgAAAAAElFTkSuQmCC\",\"orcid\":\"\",\"institution\":\"Vellore Institute of Technology University\",\"correspondingAuthor\":true,\"prefix\":\"\",\"firstName\":\"Srinivasa\",\"middleName\":\"Rao\",\"lastName\":\"Ummity\",\"suffix\":\"\"},{\"id\":489255941,\"identity\":\"42b0d08d-01da-4552-9c67-ffaa6f460a07\",\"order_by\":4,\"name\":\"Jayaram Balabaskaran\",\"email\":\"\",\"orcid\":\"\",\"institution\":\"Vellore Institute of Technology University\",\"correspondingAuthor\":false,\"prefix\":\"\",\"firstName\":\"Jayaram\",\"middleName\":\"\",\"lastName\":\"Balabaskaran\",\"suffix\":\"\"}],\"badges\":[],\"createdAt\":\"2025-05-15 11:53:10\",\"currentVersionCode\":1,\"declarations\":\"\",\"doi\":\"10.21203/rs.3.rs-6672315/v1\",\"doiUrl\":\"https://doi.org/10.21203/rs.3.rs-6672315/v1\",\"draftVersion\":[],\"editorialEvents\":[],\"editorialNote\":\"\",\"failedWorkflow\":false,\"files\":[{\"id\":90090042,\"identity\":\"edff9764-63c2-4ccd-b4bb-562258d041b8\",\"added_by\":\"auto\",\"created_at\":\"2025-08-28 11:02:01\",\"extension\":\"pdf\",\"order_by\":1,\"title\":\"\",\"display\":\"\",\"copyAsset\":false,\"role\":\"manuscript-pdf\",\"size\":352557,\"visible\":true,\"origin\":\"\",\"legend\":\"\",\"description\":\"\",\"filename\":\"Sriya.pdf\",\"url\":\"https://assets-eu.researchsquare.com/files/rs-6672315/v1_covered_45683311-87c2-4480-b416-9295e623e473.pdf\"}],\"financialInterests\":\"No competing interests reported.\",\"formattedTitle\":\"Exploring Semanticity-Based Clustering of Text Using Transformer Models: Advancing AI Applications in Education and Beyond\",\"fulltext\":[],\"fulltextSource\":\"\",\"fullText\":\"\",\"funders\":[],\"hasAdminPriorityOnWorkflow\":false,\"hasManuscriptDocX\":false,\"hasOptedInToPreprint\":true,\"hasPassedJournalQc\":\"\",\"hasAnyPriority\":false,\"hideJournal\":true,\"highlight\":\"\",\"institution\":\"\",\"isAcceptedByJournal\":false,\"isAuthorSuppliedPdf\":true,\"isDeskRejected\":\"\",\"isHiddenFromSearch\":false,\"isInQc\":false,\"isInWorkflow\":false,\"isPdf\":true,\"isPdfUpToDate\":true,\"isWithdrawnOrRetracted\":false,\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"researchsquare\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":true,\"externalIdentity\":\"\",\"sideBox\":\"\",\"snPcode\":\"\",\"submissionUrl\":\"/submission\",\"title\":\"Research Square\",\"twitterHandle\":\"researchsquare\",\"acdcEnabled\":true,\"dfaEnabled\":false,\"editorialSystem\":\"\",\"reportingPortfolio\":\"\",\"inReviewEnabled\":false,\"inReviewRevisionsEnabled\":true},\"keywords\":\"Semantic Clustering, Transformer Models, BERT and SciBERT, Educational Applications and Text Embedding and Pooling\",\"lastPublishedDoi\":\"10.21203/rs.3.rs-6672315/v1\",\"lastPublishedDoiUrl\":\"https://doi.org/10.21203/rs.3.rs-6672315/v1\",\"license\":{\"name\":\"CC BY 4.0\",\"url\":\"https://creativecommons.org/licenses/by/4.0/\"},\"manuscriptAbstract\":\"\\u003cp\\u003eThe study explores semantic-based clustering using transformer models to overcome the limitations of traditional text clustering approaches. While conventional methods rely on word frequency, this research leverages BERT and SciBERT's contextual understanding capabilities for more nuanced text organization. The methodology combines transformer-based semantic embeddings with various pooling strategies and clustering algorithms, comparing their performance against TF-IDF baselines. Experiments extended across five diverse domains: news, research papers, e-commerce products, movies, and job postings. It was observed that transformer-based embeddings with CLS pooling consistently outperformed traditional methods, producing more coherent clusters across all domains. SciBERT proved to be particularly useful for scientific text. These findings show possible applications in personalized learning systems, content organization, and recommender systems where semantic interpretation is critical. The research provides a framework to develop text clustering solutions better suited to capture contextual linkages and semantic intricacies in complex document collections.\\u003c/p\\u003e\",\"manuscriptTitle\":\"Exploring Semanticity-Based Clustering of Text Using Transformer Models: Advancing AI Applications in Education and Beyond\",\"msid\":\"\",\"msnumber\":\"\",\"nonDraftVersions\":[{\"code\":1,\"date\":\"2025-07-24 08:21:29\",\"doi\":\"10.21203/rs.3.rs-6672315/v1\",\"editorialEvents\":[{\"type\":\"communityComments\",\"content\":0}],\"status\":\"published\",\"journal\":{\"display\":true,\"email\":\"info@researchsquare.com\",\"identity\":\"researchsquare\",\"isNatureJournal\":false,\"hasQc\":true,\"allowDirectSubmit\":true,\"externalIdentity\":\"\",\"sideBox\":\"\",\"snPcode\":\"\",\"submissionUrl\":\"/submission\",\"title\":\"Research Square\",\"twitterHandle\":\"researchsquare\",\"acdcEnabled\":true,\"dfaEnabled\":false,\"editorialSystem\":\"\",\"reportingPortfolio\":\"\",\"inReviewEnabled\":false,\"inReviewRevisionsEnabled\":true}}],\"origin\":\"\",\"ownerIdentity\":\"cc3bf27b-781a-448d-85e0-491597a37e2a\",\"owner\":[],\"postedDate\":\"July 24th, 2025\",\"published\":true,\"recentEditorialEvents\":[],\"rejectedJournal\":[],\"revision\":\"\",\"amendment\":\"\",\"status\":\"posted\",\"subjectAreas\":[{\"id\":51941525,\"name\":\"Biological sciences/Computational biology and bioinformatics/Machine learning\"},{\"id\":51941526,\"name\":\"Physical sciences/Mathematics and computing\"},{\"id\":51941527,\"name\":\"Physical sciences/Mathematics and computing/Computer science\"}],\"tags\":[],\"updatedAt\":\"2025-08-28T10:53:52+00:00\",\"versionOfRecord\":[],\"versionCreatedAt\":\"2025-07-24 08:21:29\",\"video\":\"\",\"vorDoi\":\"\",\"vorDoiUrl\":\"\",\"workflowStages\":[]},\"version\":\"v1\",\"identity\":\"rs-6672315\",\"journalConfig\":\"researchsquare\"},\"__N_SSP\":true},\"page\":\"/article/[identity]/[[...version]]\",\"query\":{\"redirect\":\"/article/rs-6672315\",\"identity\":\"rs-6672315\",\"version\":[\"v1\"]},\"buildId\":\"8U1c8b4HqxoKbykW_rLl7\",\"isFallback\":false,\"isExperimentalCompile\":false,\"dynamicIds\":[84888],\"gssp\":true,\"scriptLoader\":[]}","source_license":"CC-BY-4.0","license_restricted":false}