[CVPR 2021 Oral] ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis

Overview

ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis

ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis

[arxiv|pdf|video|webpage]

Yinan He, Bei Gan, Siyu Chen, Yichun Zhou, Guojun Yin, Luchuan Song, Lu Sheng, Jing Shao, Ziwei Liu

In CVPR 2021

Abstract: The rapid progress of photorealistic synthesis techniques has reached at a critical point where the boundary between real and manipulated images starts to blur. Thus, benchmarking and advancing digital forgery analysis have become a pressing issue. However, existing face forgery datasets either have limited diversity or only support coarse-grained analysis. To counter this emerging threat, we construct the ForgeryNet dataset, an extremely large face forgery dataset with unified annotations in image- and video-level data across four tasks: 1) Image Forgery Classification, including two-way (real / fake), three-way (real / fake with identity-replaced forgery approaches / fake with identity-remained forgery approaches), and n-way (real and 15 respective forgery approaches) classification. 2) Spatial Forgery Localization, which segments the manipulated area of fake images compared to their corresponding source real images. 3) Video Forgery Classification, which re-defines the video-level forgery classification with manipulated frames in random positions. This task is important because attackers in real world are free to manipulate any target frame. and 4) Temporal Forgery Localization, to localize the temporal segments which are manipulated. ForgeryNet is by far the largest publicly available deep face forgery dataset in terms of data-scale (2.9 million images, 221,247 videos), manipulations (7 image-level approaches, 8 video-level approaches), perturbations (36 independent and more mixed perturbations) and annotations (6.3 million classification labels, 2.9 million manipulated area annotations and 221,247 temporal forgery segment labels). We perform extensive benchmarking and studies of existing face forensics methods and obtain several valuable observations.

Dataset is coming soon.


License and Citation

The use of this software is RESTRICTED to non-commercial research and educational purposes.

@article{he2021forgerynet,
  title={ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis},
  author={He, Yinan and Gan, Bei and Chen, Siyu and Zhou, Yichun and Yin, Guojun and Song, Luchuan and Sheng, Lu and Shao, Jing and Liu, Ziwei},
  journal={arXiv preprint arXiv:2103.05630},
  year={2021}
}
Comments
  • How can I open Training.tar.4 file?

    How can I open Training.tar.4 file?

    Hello, you did a great job with Forgerynet.

    I requested and obtained access to the Training folder, I downloaded one of the files contained, but I can't open it because of its extension. "Training.tar.4" for example, how can i open this folder as an ".tar"?

    opened by inglis92 7
  • Will the bounding box information be provided?

    Will the bounding box information be provided?

    Hi, thanks for the amazing work! I checked several samples of the image subset and found that sometimes there is more than one face in an image. Will you provide the exact face bounding box coordinates for image-level detection? Or could you please provide more information about the face detector you used in this work? Thanks a lot!

    opened by XJay18 5
  • Question about the label

    Question about the label

    发现数据集中一些数据标注与标签文件不同。 2 train_video_release/1/987080cdf7ec51e4c46c1055d6cdd0eb/47d69f53ec52643354975550e39130be/video.mp4 79 17 1 96 64 0 1 2 1 在数据集中发现在79帧以前还有一些数据是操纵过了的 但是没有标记,能解释一下数据标注吗?还是训练过程中只用标注过的数据?

    opened by Elijah-Yi 4
  • How to download the dataset via wget?

    How to download the dataset via wget?

    Good morning. I am trying to download the training and validation set. I got access to Google Drive but I need to be able to download it via wget. I can't because authentication is required and then via wget I am redirected to the login page. I have also tried using the 105 site but I cannot register because they do not accept Italian phone numbers.

    Do you have any alternatives to allow me to download via wget?

    opened by davide-coccomini 3
  • Downloading validation set

    Downloading validation set

    It appears that the validation (and training) set is now stored in a different location. I'm trying to download it from https://www.aliyundrive.com/s/M1FWYLZs4HJ/folder/611f0735642e1e9cf16c4e5d8d74305891be45fb, but because the site is not in English and requires an account, it was not possible to download it.

    Would it be possible to move at least the validation set to somewhere else, e.g., GoogleDrive?

    opened by ahaliassos 3
  • How do we download the dataset?

    How do we download the dataset?

    Hi,

    I tried downloading the dataset from here https://yinanhe.github.io/projects/forgerynet.html, but I couldn't find the "Download * Set" button to click. Could you help me out please?

    Many thanks!

    opened by ahaliassos 3
  • Google drive access to dataset

    Google drive access to dataset

    Hello,

    I'd appreciate if you could share your google drive dataset with me, my team and I are working on a Deepfake detection research project. My gmail is [email protected] . Thank you.

    opened by dragosconst 2
  • Provide detailed data augmentaion setup or codes mentioned in paper?

    Provide detailed data augmentaion setup or codes mentioned in paper?

    Hi, I think you add lots of data augmentation to the validation set. This results in a large gap between the training and validation sets. This can lead to a situation where a model performs well, possibly simply due to a closer augmentation setup used in the validation set. It is very unfair when compared methods on the validation set. So, I suggest you provide detailed data augmentation setup or codes mentioned in the paper.

    opened by LightningChan 2
  • tar:Unexpected EOF in archive tar: Error is not recoverable: exiting now

    tar:Unexpected EOF in archive tar: Error is not recoverable: exiting now

    Hi,I have downloaded the dataset from google drive. But when I tried to unzip the Training.tar.0 by :

    tar -xvf Training.tar.0

    the error comes as follow: tar:Unexpected EOF in archive tar:Unexpected EOF in archive tar: Error is not recoverable: exiting now and when I try to unzip the Training.tar.2:

    image image

    Why this happened? How could I get the data? Whether the files in google drive broken or not? Please give me some advice, thank you.

    opened by damengdameng 2
  • Baseline model of Xception download

    Baseline model of Xception download

    I download the model from google drive.When unzipping, a message is displayed indicating that the file is corrupted. Can you give me a new download link? For example, Baidu network disk.

    opened by yunzaitianshang 2
  • how to get the face crop as yours?

    how to get the face crop as yours?

    I want to do some experiment to reproduce the task "Spatial Forgery Localization", but I don't know how to get the same face crop as yours while the face crop area is related to the final IoU results.

    opened by niyunsheng 2
  • Data Unzipping issue

    Data Unzipping issue

    Hi,

    Thanks for this excellent dataset. I would like to know how we should unzip the dataset? For the training set we get 5 files named as Training.tar.0 ... Training.tar.4.

    When I unzip the files I don't get the directories containing fake data samples. I only get folders named, 18, 19 and 1.

    Am I missing something here? Could you please guide me on how I should extract the data?

    Thanks

    opened by sohailahmedkhan 1
  • About label list

    About label list

    Hello, thanks for sharing us such excellent work! Here i have some issues about the label list: taking a video forgery as example, "3 val_video_release/4/7f89846f0181198f2aee02c863c2aca5/718e0e03ab494625ccd35e839deafadb/video.mp4 0 10 0 10 35 1 45 30 0 1 2 4", what does these suffix numbers mean? "0 10 0 10 35 1 45 30 0 1 2 4"

    opened by Mark-Dou 0
  • Weights for SlowFast or X3D-M

    Weights for SlowFast or X3D-M

    Hello! You provided the weights for XceptionNet working on images. Can you please make public also the weights of SlowFast and X3D-M trained on the videos?

    Thank you!

    opened by davide-coccomini 0
  • Google drive dataset Error 404: File not found

    Google drive dataset Error 404: File not found

    Hi,

    we can open ForgeryNet in Chrome browser:https://drive.google.com/drive/folders/1gViqLc4l9sT1WTB-_hlcDUCWYskT2_LE, but I cannot download with gdrive tool, it shows ERROR as following:

    Failed to get file: googleapi: Error 404: File not found: 1gViqLc4l9sT1WTB-_hlcDUCWYskT2_LE., notFound

    When I try to get link of each tar file, it shows Restriced access: Only people with access can open with the link. Can you check this error and help me~ Thanks @yinanhe

    opened by reallm 1
  • Missing videos in training set list txt

    Missing videos in training set list txt

    Although there are videos in the training set manipulated by methods 2 and 8, these are not reported in the video_list.txt. I have added them by generating them within a new video_list.txt file, but I do not have the time segmentation information. Could you provide it?

    In the meantime, if anyone needs it, here is the modified file including videos 2 and 8 with the information we have available: video_list_complete.txt

    opened by davide-coccomini 2
Owner
Yinan He
Yinan He
[Official] Exploring Temporal Coherence for More General Video Face Forgery Detection(ICCV 2021)

Exploring Temporal Coherence for More General Video Face Forgery Detection(FTCN) Yinglin Zheng, Jianmin Bao, Dong Chen, Ming Zeng, Fang Wen Accepted b

null 57 Dec 28, 2022
Open-L2O: A Comprehensive and Reproducible Benchmark for Learning to Optimize Algorithms

Open-L2O This repository establishes the first comprehensive benchmark efforts of existing learning to optimize (L2O) approaches on a number of proble

VITA 161 Jan 2, 2023
[Preprint] "Bag of Tricks for Training Deeper Graph Neural Networks A Comprehensive Benchmark Study" by Tianlong Chen*, Kaixiong Zhou*, Keyu Duan, Wenqing Zheng, Peihao Wang, Xia Hu, Zhangyang Wang

Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Study Codes for [Preprint] Bag of Tricks for Training Deeper Graph

VITA 101 Dec 29, 2022
A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains (IJCV submission)

wsss-analysis The code of: A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains, arXiv pre-print 2019 paper.

Lyndon Chan 48 Dec 18, 2022
Official pytorch implementation of paper "Inception Convolution with Efficient Dilation Search" (CVPR 2021 Oral).

IC-Conv This repository is an official implementation of the paper Inception Convolution with Efficient Dilation Search. Getting Started Download Imag

Jie Liu 111 Dec 31, 2022
Official PyTorch implementation of RobustNet (CVPR 2021 Oral)

RobustNet (CVPR 2021 Oral): Official Project Webpage Codes and pretrained models will be released soon. This repository provides the official PyTorch

Sungha Choi 173 Dec 21, 2022
Code for "NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video", CVPR 2021 oral

NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video Project Page | Paper NeuralRecon: Real-Time Coherent 3D Reconstruction from Mon

ZJU3DV 1.4k Dec 30, 2022
Dynamic Slimmable Network (CVPR 2021, Oral)

Dynamic Slimmable Network (DS-Net) This repository contains PyTorch code of our paper: Dynamic Slimmable Network (CVPR 2021 Oral). Architecture of DS-

Changlin Li 197 Dec 9, 2022
[CVPR 2021 Oral] Variational Relational Point Completion Network

VRCNet: Variational Relational Point Completion Network This repository contains the PyTorch implementation of the paper: Variational Relational Point

PL 121 Dec 12, 2022
Code for "Single-view robot pose and joint angle estimation via render & compare", CVPR 2021 (Oral).

Single-view robot pose and joint angle estimation via render & compare Yann Labbé, Justin Carpentier, Mathieu Aubry, Josef Sivic CVPR: Conference on C

Yann Labbé 51 Oct 14, 2022
Quasi-Dense Similarity Learning for Multiple Object Tracking, CVPR 2021 (Oral)

Quasi-Dense Tracking This is the offical implementation of paper Quasi-Dense Similarity Learning for Multiple Object Tracking. We present a trailer th

ETH VIS Research Group 327 Dec 27, 2022
Pytorch implementation for "Adversarial Robustness under Long-Tailed Distribution" (CVPR 2021 Oral)

Adversarial Long-Tail This repository contains the PyTorch implementation of the paper: Adversarial Robustness under Long-Tailed Distribution, CVPR 20

Tong WU 89 Dec 15, 2022
Code for CVPR 2021 oral paper "Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts"

Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts The rapid progress in 3D scene understanding has come with growing dem

Facebook Research 182 Dec 30, 2022
TAP: Text-Aware Pre-training for Text-VQA and Text-Caption, CVPR 2021 (Oral)

TAP: Text-Aware Pre-training TAP: Text-Aware Pre-training for Text-VQA and Text-Caption by Zhengyuan Yang, Yijuan Lu, Jianfeng Wang, Xi Yin, Dinei Flo

Microsoft 61 Nov 14, 2022
Code for "Reconstructing 3D Human Pose by Watching Humans in the Mirror", CVPR 2021 oral

Reconstructing 3D Human Pose by Watching Humans in the Mirror Qi Fang*, Qing Shuai*, Junting Dong, Hujun Bao, Xiaowei Zhou CVPR 2021 Oral The videos a

ZJU3DV 178 Dec 13, 2022
Official PyTorch Implementation of Convolutional Hough Matching Networks, CVPR 2021 (oral)

Convolutional Hough Matching Networks This is the implementation of the paper "Convolutional Hough Matching Network" by J. Min and M. Cho. Implemented

Juhong Min 70 Nov 22, 2022
Implementation of "Bidirectional Projection Network for Cross Dimension Scene Understanding" CVPR 2021 (Oral)

Bidirectional Projection Network for Cross Dimension Scene Understanding CVPR 2021 (Oral) [ Project Webpage ] [ arXiv ] [ Video ] Existing segmentatio

Hu Wenbo 135 Dec 26, 2022
Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion (CVPR'2021, Oral)

DSA^2 F: Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion (CVPR'2021, Oral) This repo is the official imp

如今我已剑指天涯 46 Dec 21, 2022
This is the code for CVPR 2021 oral paper: Jigsaw Clustering for Unsupervised Visual Representation Learning

JigsawClustering Jigsaw Clustering for Unsupervised Visual Representation Learning Pengguang Chen, Shu Liu, Jiaya Jia Introduction This project provid

DV Lab 73 Sep 18, 2022