3CBench: A Unified Benchmarking Framework for the Computing Capacity of Heterogeneous AI Clusters

3CBench: A Unified Benchmarking Framework for the Computing Capacity of Heterogeneous AI Clusters | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article 3CBench: A Unified Benchmarking Framework for the Computing Capacity of Heterogeneous AI Clusters Weixing Zhang, Xizhi Wang, Jun Yan, Jiasun Feng, Yiying Liu, Haiyan Li, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7602328/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract The rapid evolution of Artificial Intelligence has driven the demand for extensivecomputational resources and the deployment of AI tasks across heterogeneouscomputing platforms. However, existing benchmarking systems face several chal-lenges, including limited compatibility with diverse hardware, insufficient supportfor varied deep learning frameworks and tasks, and a lack of comprehensive eval-uation metrics for the computing capacities. To address these issues, we propose3CBench, a unified benchmarking framework designed for heterogeneous AI clus-ters. Featuring a modular architecture encompassing environment management,task execution, and metrics analysis, 3CBench provides automated workflows andensures seamless compatibility with diverse GPU architectures and deep learningframeworks. It provides a comprehensive evaluation metrics system to rigorouslyassess computational performance and stability across both transformer-basedlarge language models and convolutional neural networks, thereby covering dom-inant deep learning architectures. Extensive experiments demonstrate 3CBench’sscalability on heterogeneous AI clusters, compatibility with various deep learningframeworks and tasks, and the support for a wide range of applications. Addition-ally, 3CBench aids in problem diagnosis during the development process of GPUvendors. These features establish 3CBench as a robust tool for benchmarking,optimization, and system-level evaluation in heterogeneous AI clusters. Benchmarking Framework Evaluation Metrics Automated Evaluation Scalability Computing Capacity Evaluation Metric Calculation Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7602328","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":526431799,"identity":"0c3d3c65-1527-4b7a-a2cf-a17bcedcd959","order_by":0,"name":"Weixing Zhang","email":"","orcid":"","institution":"Zhejiang Lab","correspondingAuthor":false,"prefix":"","firstName":"Weixing","middleName":"","lastName":"Zhang","suffix":""},{"id":526431800,"identity":"915a68db-6de6-4ee1-97b9-e0648470020d","order_by":1,"name":"Xizhi Wang","email":"","orcid":"","institution":"Zhejiang Lab","correspondingAuthor":false,"prefix":"","firstName":"Xizhi","middleName":"","lastName":"Wang","suffix":""},{"id":526431801,"identity":"52256c4d-6b7a-4a8f-90e4-1a8e6cc21ac8","order_by":2,"name":"Jun Yan","email":"","orcid":"","institution":"Zhejiang Lab","correspondingAuthor":false,"prefix":"","firstName":"Jun","middleName":"","lastName":"Yan","suffix":""},{"id":526431802,"identity":"fc5ffbed-3768-4edf-abf2-84b62369cf7a","order_by":3,"name":"Jiasun Feng","email":"","orcid":"","institution":"Zhejiang Lab","correspondingAuthor":false,"prefix":"","firstName":"Jiasun","middleName":"","lastName":"Feng","suffix":""},{"id":526431803,"identity":"e4bf8491-3e5f-4d1e-9c41-fd4383e24b97","order_by":4,"name":"Yiying Liu","email":"","orcid":"","institution":"Zhejiang Lab","correspondingAuthor":false,"prefix":"","firstName":"Yiying","middleName":"","lastName":"Liu","suffix":""},{"id":526431804,"identity":"541d1c39-5ffb-4b1c-a38d-063dc2610f48","order_by":5,"name":"Haiyan Li","email":"","orcid":"","institution":"Zhejiang Lab","correspondingAuthor":false,"prefix":"","firstName":"Haiyan","middleName":"","lastName":"Li","suffix":""},{"id":526431805,"identity":"ffe81ca9-2fec-4eb7-bdb3-01b1fe6845d2","order_by":6,"name":"Qun Chen","email":"","orcid":"","institution":"Zhejiang Lab","correspondingAuthor":false,"prefix":"","firstName":"Qun","middleName":"","lastName":"Chen","suffix":""},{"id":526431806,"identity":"25b48ca8-85fa-46ce-a790-1aed90d41257","order_by":7,"name":"Zhe Tang","email":"","orcid":"","institution":"Zhejiang Lab","correspondingAuthor":false,"prefix":"","firstName":"Zhe","middleName":"","lastName":"Tang","suffix":""},{"id":526431807,"identity":"c1d00bdc-db06-46d0-975f-35bf0af68eec","order_by":8,"name":"Xin Cui","email":"","orcid":"","institution":"Zhejiang Lab","correspondingAuthor":false,"prefix":"","firstName":"Xin","middleName":"","lastName":"Cui","suffix":""},{"id":526431808,"identity":"1428c0a3-fa9c-4a99-8cf8-3c9fbfa33984","order_by":9,"name":"Fei Yang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABDUlEQVRIiWNgGAWjYLCCigqGBAaGgw1ApgWDAVFazpyBa5EgUsvZNpAWMCBCi8Hxs4dfHJxXl8fPeLjxc8EvCXlzBuZjH7/g03ImL83i4Da2YsmGg83SM/skDHc2sCXPlsGn5UCOmfHHbTyJGw4cbJDm7ZFg3HCAx5hZAp+W82/MDA7OkUjcf+Bg82+gFnvCWm7kGD842GCQuIHhYJs0zw+JRJAWxg94tEjeeGPGcOBYQuKMAwfbrHkbJJJ3NrMlM+PRwcB3Psf4w4GausT+Gccf3+b5Y2O7nb35MOMPPFoUDjCwQRwucYCBgbENyABawcyDR4t8AwMzxOH8DUDiD0QUry2jYBSMglEw4gAAWFpaGLr1ZpMAAAAASUVORK5CYII=","orcid":"","institution":"Zhejiang Lab","correspondingAuthor":true,"prefix":"","firstName":"Fei","middleName":"","lastName":"Yang","suffix":""}],"badges":[],"createdAt":"2025-09-12 16:23:27","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7602328/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7602328/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":93179426,"identity":"799bdbd1-0d71-4554-868d-57f531814bf9","added_by":"auto","created_at":"2025-10-09 23:29:56","extension":"json","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":10150,"visible":true,"origin":"","legend":"","description":"","filename":"b6049705f2f34638be5a486c53881a5c.json","url":"https://assets-eu.researchsquare.com/files/rs-7602328/v1/9b31f0e8f5093680a5be6f47.json"},{"id":99790193,"identity":"04478527-38ad-4918-9658-bb43341b9d98","added_by":"auto","created_at":"2026-01-08 12:57:13","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1575044,"visible":true,"origin":"","legend":"","description":"","filename":"3CBench.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7602328/v1_covered_e340d322-1f28-4d16-873b-6106eca90b2a.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"\u003cp\u003e3CBench: A Unified Benchmarking Framework for the Computing Capacity of Heterogeneous AI Clusters\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Benchmarking Framework, Evaluation Metrics, Automated Evaluation, Scalability, Computing Capacity Evaluation, Metric Calculation","lastPublishedDoi":"10.21203/rs.3.rs-7602328/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7602328/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"The rapid evolution of Artificial Intelligence has driven the demand for extensivecomputational resources and the deployment of AI tasks across heterogeneouscomputing platforms. However, existing benchmarking systems face several chal-lenges, including limited compatibility with diverse hardware, insufficient supportfor varied deep learning frameworks and tasks, and a lack of comprehensive eval-uation metrics for the computing capacities. To address these issues, we propose3CBench, a unified benchmarking framework designed for heterogeneous AI clus-ters. Featuring a modular architecture encompassing environment management,task execution, and metrics analysis, 3CBench provides automated workflows andensures seamless compatibility with diverse GPU architectures and deep learningframeworks. It provides a comprehensive evaluation metrics system to rigorouslyassess computational performance and stability across both transformer-basedlarge language models and convolutional neural networks, thereby covering dom-inant deep learning architectures. Extensive experiments demonstrate 3CBench’sscalability on heterogeneous AI clusters, compatibility with various deep learningframeworks and tasks, and the support for a wide range of applications. Addition-ally, 3CBench aids in problem diagnosis during the development process of GPUvendors. These features establish 3CBench as a robust tool for benchmarking,optimization, and system-level evaluation in heterogeneous AI clusters.","manuscriptTitle":"3CBench: A Unified Benchmarking Framework for the Computing Capacity of Heterogeneous AI Clusters","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-09 23:29:52","doi":"10.21203/rs.3.rs-7602328/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"0291a6f3-f9e7-434d-bdb9-a71bae7f7543","owner":[],"postedDate":"October 9th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-01-04T20:23:47+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-09 23:29:52","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7602328","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7602328","identity":"rs-7602328","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}