Optimized deep learning model for spatio-temporal detection and localization of object removal video forgery with multiple feature extraction

Optimized deep learning model for spatio-temporal detection and localization of object removal video forgery with multiple feature extraction | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Optimized deep learning model for spatio-temporal detection and localization of object removal video forgery with multiple feature extraction Lakshmi Kumari Ch, PRASAD K.V This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1641193/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Video forgery (VF) is an approach for manipulating the fake videos by modifying, coordinating or generating new contents among the video sequence. The identification of this type of video forgery is complex. For extracting multiple features, developing a novel approach still remains major challenge in this area. In this work, an optimized deep learning (DL) model for the video forgery detection (VFD) and localization with multiple feature extraction is proposed. Initially, key-frame extraction takes place with the aid of Gaussian mixture model (GMM) to extract frames from the forged videos. Then, pre-processing stage is manipulated for the conversion of RGB frame into grayscale image. To study the nature of the forged videos, there is a need of extracting multi-features from the pre-processed frames. In our proposed study, SURF, PCA-HOG, MBFDF, COA and PRG features are extracted. The dataset used for proposed work is collected from REWIND of about 80 forged and authenticated videos. With the help of DL approach, video forgery can be detected and localized. Thus, this research mainly focus on the detection and localization of forged video based on ResNet152V2 model hybrid with Bi-GRU to attain the maximum accuracy and efficiency. The performance of this model is finally compared with existing approachesin terms of accuracy, precision, F-measure, sensitivity, specificity, FNR, FDR, FPR, MCC and NPV. The proposed methodology assures the performance of96.17% accuracy, 96% precision, 96.14% F-measure, 96.58% sensitivity, 96.5% specificity, 0.034 FNR, 0.04 FDR, 0.034 FPR, 0.92 MCC, 96% NPV respectively. Along with is, the mean square error (MSE) and peak-to-signal-noise ratio (PSNR) for GMM model attained about 104 and 27.95 respectively. Video forgery detection and localization GMM model Pre-processing multiple features ResNet152V2 Bi-GRU Improved remora optimization Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1641193","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":105477804,"identity":"4ef7812a-2e51-45af-82e3-bc5726e7ab21","order_by":0,"name":"Lakshmi Kumari Ch","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABBElEQVRIiWNgGAWjYBACPgY2BmYGBgkDMC+hwkaOgYEHvxY2VC1n0oyJ1cIA0cLYdiixgaAW9mPJnwtzLIwNrh1+9uFh24H0DcfPHnzwgcFOTrcBhxaetGPSM7dJmBncTjOekXDuTu6GM3nJhjMYko3NDuByWHobM+82CRuD2wnGDAllz3I3HMgxk+ZhOJC4DZcW/ufNnyFa0j8zJLAdTjc4/4aAFom0A9K8YIflAG1pO5xgcIOQLRLP0kBajCVv5xSDAtlw5o03xoYzDHD7hZ8/zRjosDrDvtvpmxl/VNjI853PMXzwocJODpcWTKAAVmlArHIQkG8gRfUoGAWjYBSMBAAA+apc+mpGUS4AAAAASUVORK5CYII=","orcid":"","institution":"Koneru Lakshmaiah Education Foundation","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Lakshmi","middleName":"Kumari","lastName":"Ch","suffix":""},{"id":105477805,"identity":"3193380f-fd91-41c2-850c-21654bbaff8d","order_by":1,"name":"PRASAD K.V","email":"","orcid":"","institution":"Koneru Lakshmaiah Education Foundation","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"PRASAD","middleName":"","lastName":"K.V","suffix":""}],"badges":[],"createdAt":"2022-05-10 09:29:28","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1641193/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1641193/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":21830092,"identity":"ec8a3853-c835-43f2-900c-d54895d4e1b0","added_by":"auto","created_at":"2022-05-24 15:45:39","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1021416,"visible":true,"origin":"","legend":"","description":"","filename":"Paper.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1641193/v1_covered.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Optimized deep learning model for spatio-temporal detection and localization of object removal video forgery with multiple feature extraction","fulltext":[{"header":"Full Text","content":"This preprint is available for \u003ca href='/article/rs-1641193/latest.pdf' target='_blank'\u003edownload as a PDF\u003c/a\u003e."}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Video forgery detection and localization, GMM model, Pre-processing, multiple features, ResNet152V2, Bi-GRU, Improved remora optimization","lastPublishedDoi":"10.21203/rs.3.rs-1641193/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1641193/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eVideo forgery (VF) is an approach for manipulating the fake videos by modifying, coordinating or generating new contents among the video sequence. The identification of this type of video forgery is complex. For extracting multiple features, developing a novel approach still remains major challenge in this area. In this work, an optimized deep learning (DL) model for the video forgery detection (VFD) and localization with multiple feature extraction is proposed. Initially, key-frame extraction takes place with the aid of Gaussian mixture model (GMM) to extract frames from the forged videos. Then, pre-processing stage is manipulated for the conversion of RGB frame into grayscale image. To study the nature of the forged videos, there is a need of extracting multi-features from the pre-processed frames. In our proposed study, SURF, PCA-HOG, MBFDF, COA and PRG features are extracted. The dataset used for proposed work is collected from REWIND of about 80 forged and authenticated videos. With the help of DL approach, video forgery can be detected and localized. Thus, this research mainly focus on the detection and localization of forged video based on ResNet152V2 model hybrid with Bi-GRU to attain the maximum accuracy and efficiency. The performance of this model is finally compared with existing approachesin terms of accuracy, precision, F-measure, sensitivity, specificity, FNR, FDR, FPR, MCC and NPV. The proposed methodology assures the performance of96.17% accuracy, 96% precision, 96.14% F-measure, 96.58% sensitivity, 96.5% specificity, 0.034 FNR, 0.04 FDR, 0.034 FPR, 0.92 MCC, 96% NPV respectively. Along with is, the mean square error (MSE) and peak-to-signal-noise ratio (PSNR) for GMM model attained about 104 and 27.95 respectively.\u003c/p\u003e","manuscriptTitle":"Optimized deep learning model for spatio-temporal detection and localization of object removal video forgery with multiple feature extraction","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-05-24 15:45:31","doi":"10.21203/rs.3.rs-1641193/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2fb4cedc-491c-4a0f-9ed1-07899c6cfb60","owner":[],"postedDate":"May 24th, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2023-01-25T23:44:14+00:00","versionOfRecord":[],"versionCreatedAt":"2022-05-24 15:45:31","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-1641193","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1641193","identity":"rs-1641193","version":["v1"]},"buildId":"7rjqhiLT3MXkJMwkYKINL","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}