Novel Nesting of Deep Learning Domain Transfer and Hybrid Video Coding for Video Compression

Novel Nesting of Deep Learning Domain Transfer and Hybrid Video Coding for Video Compression | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Novel Nesting of Deep Learning Domain Transfer and Hybrid Video Coding for Video Compression Shaohua Jia, Wan-Chi Siu, Pengyu Liu, Kebin Jia This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8146081/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Efficient video compression is crucial for addressing the exponential growth of video content, which now constitutes a significant portion of global internet traffic. Traditional compression standards mainly include H.264 and H.265, while the current research trend is to partially or completely replace the architectures of these traditional methods with deep learning techniques. However, these two approaches are not mutually exclusive. Based on the idea, this paper proposes a new direction that combines rhythmically traditional video compression methods with deep learning techniques to achieve higher compression efficiency and improve reconstruction quality. We adopt a two-stage compression framework, where video frames are firstly down-sized using bicubic downsampling and then encoded using traditional codecs such as H.264 or H.265. Subsequently, we employ a deep learning-based Video Super-Resolution model to restore skillfully the compressed video frames. Furthermore, it is a challenge to construct structured temporal priors at different semantic levels to better model implicitly the abstraction process from local to global representation. Aiming at this, in our Video Super-Resolution model, we have made a specially designed domain to adaptively process the structured temporal priors for different semantic levels. Besides, unlike traditional compression methods, deep learning-based compression algorithms have high demands on computational resources. Currently, most research results are unable to execute 2160P video compression tasks on a single RTX 4090. Based on this, we design a Hierarchical Simplified Attention-Net to reduce model complexity, which can perform compression tasks at resolutions up to 2160P on a single RTX 4090 GPU. Finally, our model achieves more remarkable results on benchmark datasets such as UVG, MCL-JCV, and HEVC Classes B, C, D, and E. Video Coding Video Super-Resolution Hybrid Coding Domain transfer and Attention Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8146081","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":573824339,"identity":"b06ae1db-28d2-4e06-80aa-8fca011063f1","order_by":0,"name":"Shaohua Jia","email":"","orcid":"","institution":"St. Francis University","correspondingAuthor":false,"prefix":"","firstName":"Shaohua","middleName":"","lastName":"Jia","suffix":""},{"id":573824341,"identity":"f9639441-1075-4c47-b291-959a1531f7e4","order_by":1,"name":"Wan-Chi Siu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA/klEQVRIiWNgGAWjYLACxgYJBgZmBsYHDAwWYAEg94ABMVqYDcCKidQCptgkiNJicPzsAYafOywY5Nt5zKoLKiQSNxxgPnibh+GOMU4tZ/ISGHvPSDAYHOYxuz3jDEgLW7I1D8MzM1xazA7kGDAztgG1MAO18LaBtPCYSfMwHLbBqeX8G4gW+WYes2KIFv5v+LXcgNrCAHQYM9QWNpAWnA6zv/HG4GBvmwSPwWG2YmmgX4xnHmYztpxjcBin9yX7cwwf/Gyrk5PvP7zxc0GFjWzf8eaHN95UHDZswKUHCA4AMQ+IwQzEjg0gkgF/RCIASLE9kWpHwSgYBaNgBAEAtJlPTARWtwcAAAAASUVORK5CYII=","orcid":"","institution":"St. Francis University","correspondingAuthor":true,"prefix":"","firstName":"Wan-Chi","middleName":"","lastName":"Siu","suffix":""},{"id":573824352,"identity":"7c5001f2-0359-407c-a860-02428d9b44fa","order_by":2,"name":"Pengyu Liu","email":"","orcid":"","institution":"Beijing University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Pengyu","middleName":"","lastName":"Liu","suffix":""},{"id":573824354,"identity":"37db4fe2-65d5-4323-b66f-e9e8797a1f58","order_by":3,"name":"Kebin Jia","email":"","orcid":"","institution":"Beijing University of Technology","correspondingAuthor":false,"prefix":"","firstName":"Kebin","middleName":"","lastName":"Jia","suffix":""}],"badges":[],"createdAt":"2025-11-18 13:38:18","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8146081/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8146081/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":100372400,"identity":"70334a8f-eae4-4827-b32d-d79df1446dd6","added_by":"auto","created_at":"2026-01-16 08:12:15","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":3320815,"visible":true,"origin":"","legend":"","description":"","filename":"20250925JiaSiuVideoCodingV22.docx","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/b91302259fcae7104636bc9d.docx"},{"id":100261312,"identity":"5cccc428-9ad4-4c43-943b-5b531b20af70","added_by":"auto","created_at":"2026-01-14 17:12:40","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":6294,"visible":true,"origin":"","legend":"","description":"","filename":"239d2808d8374ad4816d07c9e88d2d51.json","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/9a3fece3706aa50c3c0aa637.json"},{"id":100261314,"identity":"bf95f275-d21e-420b-a656-31fb59fd4820","added_by":"auto","created_at":"2026-01-14 17:12:40","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":149253,"visible":true,"origin":"","legend":"","description":"","filename":"239d2808d8374ad4816d07c9e88d2d511enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/6393eceee09c054d9307a281.xml"},{"id":100372757,"identity":"72f72314-cc8a-4901-8e15-ea6fd504ed54","added_by":"auto","created_at":"2026-01-16 08:13:08","extension":"jpeg","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":47037,"visible":true,"origin":"","legend":"","description":"","filename":"groupimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/c53e05da598b4d618b1fdb60.jpeg"},{"id":100261324,"identity":"d50bf962-349b-4331-ae33-5fdf3b8246d3","added_by":"auto","created_at":"2026-01-14 17:12:40","extension":"jpeg","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":46033,"visible":true,"origin":"","legend":"","description":"","filename":"groupimage2.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/60dcec080e08c0d109b35104.jpeg"},{"id":100371071,"identity":"e1d116ea-f298-479b-a327-f06a40fb4d5a","added_by":"auto","created_at":"2026-01-16 08:09:21","extension":"jpeg","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":25588,"visible":true,"origin":"","legend":"","description":"","filename":"groupimage3.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/40032615ecb29e009fbf079f.jpeg"},{"id":100261320,"identity":"576c223e-0ad9-41a6-97e6-03615333915e","added_by":"auto","created_at":"2026-01-14 17:12:40","extension":"jpeg","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":74258,"visible":true,"origin":"","legend":"","description":"","filename":"groupimage4.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/afad62481f6fb476aa5a4021.jpeg"},{"id":100261319,"identity":"d59e59f6-59b6-4099-b31c-dd8c813c8de4","added_by":"auto","created_at":"2026-01-14 17:12:40","extension":"jpeg","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":86526,"visible":true,"origin":"","legend":"","description":"","filename":"groupimage5.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/2f486c20a9dff1c61fcec69b.jpeg"},{"id":100372660,"identity":"0a8c1b18-742d-4dfd-84f7-fa630ec336a0","added_by":"auto","created_at":"2026-01-16 08:12:54","extension":"jpeg","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":87102,"visible":true,"origin":"","legend":"","description":"","filename":"groupimage6.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/b8c446be278631ba02d64991.jpeg"},{"id":100372249,"identity":"d60c7b52-f5eb-4905-a3df-a14c59c0ef9a","added_by":"auto","created_at":"2026-01-16 08:11:52","extension":"jpeg","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":20569,"visible":true,"origin":"","legend":"","description":"","filename":"groupimage7.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/a37644b76cb50e42994c2315.jpeg"},{"id":100261327,"identity":"81534c71-4c7e-4f5b-a6da-c619bc1ba73b","added_by":"auto","created_at":"2026-01-14 17:12:40","extension":"jpeg","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":21092,"visible":true,"origin":"","legend":"","description":"","filename":"groupimage8.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/0930ebbe3734f988ddf7a6cc.jpeg"},{"id":100371977,"identity":"cd2bd20c-e8d4-4987-bab0-ba692b9fe966","added_by":"auto","created_at":"2026-01-16 08:11:19","extension":"png","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":31094,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinegroupimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/626d6b91a0cb175288ef47a7.png"},{"id":100371665,"identity":"6ecce819-eebd-467c-a034-8c6f5d0f3eb0","added_by":"auto","created_at":"2026-01-16 08:10:41","extension":"png","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":12380,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinegroupimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/c5c2b8c0406562cde9edf7c5.png"},{"id":100372100,"identity":"e6fe75c1-0f7e-4281-a935-6af85cb57d13","added_by":"auto","created_at":"2026-01-16 08:11:39","extension":"png","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":16776,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinegroupimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/94c211660a8694c8b001b6f6.png"},{"id":100261328,"identity":"9594d2a1-d69e-4725-a3cb-147e631b0d8b","added_by":"auto","created_at":"2026-01-14 17:12:40","extension":"png","order_by":14,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":21532,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinegroupimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/1928c4609941496e3b12ba82.png"},{"id":100372050,"identity":"77a135aa-05ba-4ed7-94be-5ec8de14b9e1","added_by":"auto","created_at":"2026-01-16 08:11:29","extension":"png","order_by":15,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":16090,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinegroupimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/f990c1a664f4b7e85995f456.png"},{"id":100261322,"identity":"867dfba2-863e-4bca-8a11-3a50974115e4","added_by":"auto","created_at":"2026-01-14 17:12:40","extension":"png","order_by":16,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":26685,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinegroupimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/c252efc7790a62a927c3088b.png"},{"id":100261330,"identity":"8c6a9cb5-6419-4d7f-93ac-6022c62ff018","added_by":"auto","created_at":"2026-01-14 17:12:40","extension":"png","order_by":17,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":6569,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinegroupimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/0760ce3e0c05c4030a8c4059.png"},{"id":100372262,"identity":"fcf0ac5c-508e-4904-ab7d-43b2de471e59","added_by":"auto","created_at":"2026-01-16 08:11:53","extension":"png","order_by":18,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":15985,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinegroupimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/4590364d90d78cc158f66669.png"},{"id":100372109,"identity":"7965e1f2-9c6f-4734-8615-c3d097ce9b1f","added_by":"auto","created_at":"2026-01-16 08:11:41","extension":"xml","order_by":19,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":137970,"visible":true,"origin":"","legend":"","description":"","filename":"239d2808d8374ad4816d07c9e88d2d511structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/843562b3ba58bffe2e0ae841.xml"},{"id":100261332,"identity":"57370b1c-7aa7-4439-ade4-e2ea8592ef83","added_by":"auto","created_at":"2026-01-14 17:12:40","extension":"html","order_by":20,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":165674,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1/5094d8c573960fe1f1780a0d.html"},{"id":104783299,"identity":"55ffb5be-9de5-45a4-ad58-fd6925a35f2b","added_by":"auto","created_at":"2026-03-17 07:58:35","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":671212,"visible":true,"origin":"","legend":"","description":"","filename":"20250925JiaSiuVideoCodingV22.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8146081/v1_covered_4820b370-0005-4a3a-b730-65238798bbc5.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Novel Nesting of Deep Learning Domain Transfer and Hybrid Video Coding for Video Compression","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Video Coding, Video Super-Resolution, Hybrid Coding, Domain transfer and Attention","lastPublishedDoi":"10.21203/rs.3.rs-8146081/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8146081/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eEfficient video compression is crucial for addressing the exponential growth of video content, which now constitutes a significant portion of global internet traffic. Traditional compression standards mainly include H.264 and H.265, while the current research trend is to partially or completely replace the architectures of these traditional methods with deep learning techniques. However, these two approaches are not mutually exclusive. Based on the idea, this paper proposes a new direction that combines rhythmically traditional video compression methods with deep learning techniques to achieve higher compression efficiency and improve reconstruction quality. We adopt a two-stage compression framework, where video frames are firstly down-sized using bicubic downsampling and then encoded using traditional codecs such as H.264 or H.265. Subsequently, we employ a deep learning-based Video Super-Resolution model to restore skillfully the compressed video frames. Furthermore, it is a challenge to construct structured temporal priors at different semantic levels to better model implicitly the abstraction process from local to global representation. Aiming at this, in our Video Super-Resolution model, we have made a specially designed domain to adaptively process the structured temporal priors for different semantic levels. Besides, unlike traditional compression methods, deep learning-based compression algorithms have high demands on computational resources. Currently, most research results are unable to execute 2160P video compression tasks on a single RTX 4090. Based on this, we design a Hierarchical Simplified Attention-Net to reduce model complexity, which can perform compression tasks at resolutions up to 2160P on a single RTX 4090 GPU. Finally, our model achieves more remarkable results on benchmark datasets such as UVG, MCL-JCV, and HEVC Classes B, C, D, and E.\u003c/p\u003e","manuscriptTitle":"Novel Nesting of Deep Learning Domain Transfer and Hybrid Video Coding for Video Compression","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-01-14 17:12:35","doi":"10.21203/rs.3.rs-8146081/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"d7db5bb3-cfcb-4110-a27b-92c8252410df","owner":[],"postedDate":"January 14th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-03-15T21:24:06+00:00","versionOfRecord":[],"versionCreatedAt":"2026-01-14 17:12:35","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8146081","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8146081","identity":"rs-8146081","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}