Frame Duplication and Insertion Forgery Detection in Surveillance Videos Using Optical Flow and Texture Features

Abstract Surveillance cameras are widely used to provide protection and security through online tracking or investigation of stored videos of an incident. Furthermore, footage of recorded videos may be used as strong evidence in the courts of law or insurance companies, but their authenticity cannot be taken for granted. Two common video inter-frame forgery types are frame duplication (FD) and frame insertion (FI). Several techniques exist in the literature to deal with them by analyzing the abnormalities caused by these operations. However, they have limited applicability, poor generalization, and high computational complexity. To tackle these issues, we propose a robust hybrid forensic system based on the idea that FD or FI causes motion inconsistency at the start and end of duplicated/inserted frames. These inconsistencies, when analyzed in an appropriate manner, help reveal the evidence of forgery. The system encompasses two forensic techniques. The first is a novel method based on the texture of motion residual component where a motion residual-based local binary pattern histogram (MR-LBPH) and an SVM classifier with the linear kernel are employed to detect suspected tampered positions. The second component is the sum consistency of optical flow (OF) and standard deviation of MR-LBPH of consecutive frames to remove false positives and precise localization of forgery. By taking the intersection of the frames detected by the two methods, we remove the false positives and get the frames bounding the duplicated/inserted region. The methods were trained and tested on our developed large Video Tampering Evaluation Dataset (VTED) and cross-validated on publicly available datasets. Cross-dataset evaluation yielded detection accuracy above 99.5%, ensuring the proposed method’s generalization; it also precisely locates the locations of tampering. As the public datasets used for cross-dataset validation include videos of different formats and frame rates, it ensures the wide applicability of the method. Moreover, the method is computationally efficient and can be run in a matter of microseconds.