PyDeepFakeDet is an integrated and scalable tool for Deepfake detection.

Overview

PyDeepFakeDet

An integrated and scalable library for Deepfake detection research.

Introduction

PyDeepFakeDet is an integrated and scalable Deepfake detection tool developed by Fudan Vision and Learning Lab. The goal is to provide state-of-the-art Deepfake detection Models as well as interfaces for the training and evaluation of new Models on commonly used Deepfake datasets.

This repository includes implementations of both CNN-based and Transformer-based methods:

Model Zoo and Baselines

The baseline Models on three versions of FF-DF dataset are provided.

Method RAW C23 C40 Model
ResNet50 97.61 94.87 84.95 RAW / C23 / C40
Xception 97.84 95.24 86.27 RAW / C23 / C40
EfficientNet-b4 97.89 95.61 87.12 RAW / C23 / C40
Meso4 85.14 77.14 60.13 RAW / C23 / C40
MesoInception4 95.45 84.13 71.31 RAW / C23 / C40
GramNet 97.65 95.16 86.21 RAW / C23 / C40
F3Net 99.95 97.52 90.43 RAW / C23 / C40
MAT 97.90 95.59 87.06 RAW / C23 / C40
ViT 96.72 93.45 82.97 RAW / C23 / C40
M2TR 99.50 97.93 92.89 RAW / C23 / C40

The baseline Models on Celeb-DF is also available.

Method Celeb-DF Model
ResNet50 98.51 CelebDF
Xception 99.05 CelebDF
EfficientNet-b4 99.44 CelebDF
Meso4 73.04 CelebDF
MesoInception4 75.87 CelebDF
GramNet 98.67 CelebDF
F3Net 96.47 CelebDF
MAT 99.02 CelebDF
ViT 96.73 CelebDF
M2TR 99.76 CelebDF

Installation

  • We use Python == 3.9.0, torch==1.11.0, torchvision==1.12.0.

  • Install the required packages by:

    pip install -r requirements.txt

Data Preparation

Please follow the instructions in DATASET.md to prepare the data.

Quick Start

Specify the path of your local dataset in ./configs/resnet50.yaml, and then run:

python run.py --cfg resnet50.yaml

Visualization tools

Please refer to VISUALIZE.md for detailed instructions.

Contributors

PyDeepFakeDet is written and maintained by Wenhao Ouyang, Chao Zhang, Zhenxin Li, and Junke Wang.

License

PyDeepFakeDet is released under the MIT license.

Citations

@inproceedings{wang2021m2tr,
  title={M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection},
  author={Wang, Junke and Wu, Zuxuan and Ouyang, Wenhao and Han, Xintong and Chen, Jingjing and Lim, Ser-Nam and Jiang, Yu-Gang},
  booktitle={ICMR},
  year={2022}
}
Comments
  • 'EfficientNet' object has no attribute 'extract_textures'

    'EfficientNet' object has no attribute 'extract_textures'

    Hey , I compared the code with your other repository "m2tr", I found that your deleted the def extract_textures in efficientnet.py, and this caused the code to report an error —— 'EfficientNet' object has no attribute 'extract_textures'

    opened by Kidleyh 3
  • RuntimeError: Error(s) in loading state_dict for M2TR

    RuntimeError: Error(s) in loading state_dict for M2TR

    We tried to load the corresponding weights for the model but found them to be inconsistent, including c23 and c40. We don't understand why, it looks like the keys are wrong.

    RuntimeError: Error(s) in loading state_dict for M2TR: Missing key(s) in state_dict: "layers.2.0.t.attention.query_embedding.weight", "layers.2.0.t.attention.query_embedding.bias", "layers.2.0.t.attention.value_embedding.weight", "layers.2.0.t.attention.value_embedding.bias", "layers.2.0.t.attention.key_embedding.weight", "layers.2.0.t.attention.key_embedding.bias", "layers.2.0.t.attention.output_linear.0.weight", "layers.2.0.t.attention.output_linear.0.bias", "layers.2.0.t.attention.output_linear.1.weight", "layers.2.0.t.attention.output_linear.1.bias", "layers.2.0.t.attention.output_linear.1.running_mean", "layers.2.0.t.attention.output_linear.1.running_var", "layers.2.0.t.feed_forward.conv.0.weight", "layers.2.0.t.feed_forward.conv.0.bias", "layers.2.0.t.feed_forward.conv.1.weight", "layers.2.0.t.feed_forward.conv.1.bias", "layers.2.0.t.feed_forward.conv.1.running_mean", "layers.2.0.t.feed_forward.conv.1.running_var", "layers.2.0.t.feed_forward.conv.3.weight", "layers.2.0.t.feed_forward.conv.3.bias", "layers.2.0.t.feed_forward.conv.4.weight", "layers.2.0.t.feed_forward.conv.4.bias", "layers.2.0.t.feed_forward.conv.4.running_mean", "layers.2.0.t.feed_forward.conv.4.running_var", "layers.2.1.filter.complex_weight", "layers.2.1.feed_forward.conv.0.weight", "layers.2.1.feed_forward.conv.0.bias", "layers.2.1.feed_forward.conv.1.weight", "layers.2.1.feed_forward.conv.1.bias", "layers.2.1.feed_forward.conv.1.running_mean", "layers.2.1.feed_forward.conv.1.running_var", "layers.2.1.feed_forward.conv.3.weight", "layers.2.1.feed_forward.conv.3.bias", "layers.2.1.feed_forward.conv.4.weight", "layers.2.1.feed_forward.conv.4.bias", "layers.2.1.feed_forward.conv.4.running_mean", "layers.2.1.feed_forward.conv.4.running_var", "layers.2.2.conv1.weight", "layers.2.2.conv1.bias", "layers.2.2.conv2.weight", "layers.2.2.conv2.bias", "layers.2.2.conv3.weight", "layers.2.2.conv3.bias", "layers.2.2.conv4.0.weight", "layers.2.2.conv4.0.bias", "layers.2.2.conv4.1.weight", "layers.2.2.conv4.1.bias", "layers.2.2.conv4.1.running_mean", "layers.2.2.conv4.1.running_var", "layers.3.0.t.attention.query_embedding.weight", "layers.3.0.t.attention.query_embedding.bias", "layers.3.0.t.attention.value_embedding.weight", "layers.3.0.t.attention.value_embedding.bias", "layers.3.0.t.attention.key_embedding.weight", "layers.3.0.t.attention.key_embedding.bias", "layers.3.0.t.attention.output_linear.0.weight", "layers.3.0.t.attention.output_linear.0.bias", "layers.3.0.t.attention.output_linear.1.weight", "layers.3.0.t.attention.output_linear.1.bias", "layers.3.0.t.attention.output_linear.1.running_mean", "layers.3.0.t.attention.output_linear.1.running_var", "layers.3.0.t.feed_forward.conv.0.weight", "layers.3.0.t.feed_forward.conv.0.bias", "layers.3.0.t.feed_forward.conv.1.weight", "layers.3.0.t.feed_forward.conv.1.bias", "layers.3.0.t.feed_forward.conv.1.running_mean", "layers.3.0.t.feed_forward.conv.1.running_var", "layers.3.0.t.feed_forward.conv.3.weight", "layers.3.0.t.feed_forward.conv.3.bias", "layers.3.0.t.feed_forward.conv.4.weight", "layers.3.0.t.feed_forward.conv.4.bias", "layers.3.0.t.feed_forward.conv.4.running_mean", "layers.3.0.t.feed_forward.conv.4.running_var", "layers.3.1.filter.complex_weight", "layers.3.1.feed_forward.conv.0.weight", "layers.3.1.feed_forward.conv.0.bias", "layers.3.1.feed_forward.conv.1.weight", "layers.3.1.feed_forward.conv.1.bias", "layers.3.1.feed_forward.conv.1.running_mean", "layers.3.1.feed_forward.conv.1.running_var", "layers.3.1.feed_forward.conv.3.weight", "layers.3.1.feed_forward.conv.3.bias", "layers.3.1.feed_forward.conv.4.weight", "layers.3.1.feed_forward.conv.4.bias", "layers.3.1.feed_forward.conv.4.running_mean", "layers.3.1.feed_forward.conv.4.running_var", "layers.3.2.conv1.weight", "layers.3.2.conv1.bias", "layers.3.2.conv2.weight", "layers.3.2.conv2.bias", "layers.3.2.conv3.weight", "layers.3.2.conv3.bias", "layers.3.2.conv4.0.weight", "layers.3.2.conv4.0.bias", "layers.3.2.conv4.1.weight", "layers.3.2.conv4.1.bias", "layers.3.2.conv4.1.running_mean", "layers.3.2.conv4.1.running_var", "classifier.projection.weight", "classifier.projection.bias". Unexpected key(s) in state_dict: "classifier.weight", "classifier.bias". size mismatch for model._fc.weight: copying a param with shape torch.Size([1, 1792]) from checkpoint, the shape in current model is torch.Size([2, 1792]). size mismatch for model._fc.bias: copying a param with shape torch.Size([1]) from checkpoint, the shape in current model is torch.Size([2]).

    Hope this can be fixed soon.

    opened by zhaoyuchen803 1
  • RuntimeError: CUDA error: invalid device ordinal

    RuntimeError: CUDA error: invalid device ordinal

    RuntimeError: CUDA error: invalid device ordinal CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Excuse me, how to solve this error.

    opened by jishenghuang 1
  • Training stuck for MAT

    Training stuck for MAT

    when I training MAT, The training is stuck, and it's not over. I checked a lot, but I didn't find the relevant information. image

    image

    image

    CPU shows running, Do you know what the problem is?

    image

    opened by Elijah-Yi 0
  • about the reproduced results

    about the reproduced results

    Hi,

    Thanks for your excellent summary. I have reproduced the related results on the dataset "raw" of FF-DF. But I got the results as shown: image image The overall values are on the high side, where meso4, MesoInception4 is not the correct one.

    PS: My dataset where the original is real, manipulated is fake, where fake and real each have 1000 videos * 128 frames. I follow your default setup of the code for extracting frames from the video. Is it normal? Or is there something I ignore?

    thanks~

    opened by twowwj 0
Owner
Junke, Wang
I'm a first-year Ph.D. student in the school of computer science at Fudan University, supervised by Prof. Zuxuan Wu and Prof. Yu-Gang Jiang.
Junke, Wang
A lane detection integrated Real-time Instance Segmentation based on YOLACT (You Only Look At CoefficienTs)

Real-time Instance Segmentation and Lane Detection This is a lane detection integrated Real-time Instance Segmentation based on YOLACT (You Only Look

Jin 4 Dec 30, 2022
(JMLR'19) A Python Toolbox for Scalable Outlier Detection (Anomaly Detection)

Python Outlier Detection (PyOD) Deployment & Documentation & Stats Build Status & Coverage & Maintainability & License PyOD is a comprehensive and sca

Yue Zhao 6.6k Jan 3, 2023
The deployment framework aims to provide a simple, lightweight, fast integrated, pipelined deployment framework that ensures reliability, high concurrency and scalability of services.

savior是一个能够进行快速集成算法模块并支持高性能部署的轻量开发框架。能够帮助将团队进行快速想法验证(PoC),避免重复的去github上找模型然后复现模型;能够帮助团队将功能进行流程拆解,很方便的提高分布式执行效率;能够有效减少代码冗余,减少不必要负担。

Tao Luo 125 Dec 22, 2022
ivadomed is an integrated framework for medical image analysis with deep learning.

Repository on the collaborative IVADO medical imaging project between the Mila and NeuroPoly labs.

null 144 Dec 19, 2022
Discretized Integrated Gradients for Explaining Language Models (EMNLP 2021)

Discretized Integrated Gradients for Explaining Language Models (EMNLP 2021) Overview of paths used in DIG and IG. w is the word being attributed. The

INK Lab @ USC 17 Oct 27, 2022
Official PyTorch implementation of "AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks"

AASIST This repository provides the overall framework for training and evaluating audio anti-spoofing systems proposed in 'AASIST: Audio Anti-Spoofing

Clova AI Research 56 Jan 2, 2023
Examples of using f2py to get high-speed Fortran integrated with Python easily

f2py Examples Simple examples of using f2py to get high-speed Fortran integrated with Python easily. These examples are also useful to troubleshoot pr

Michael 35 Aug 21, 2022
PyTorch implementation of 'Gen-LaneNet: a generalized and scalable approach for 3D lane detection'

(pytorch) Gen-LaneNet: a generalized and scalable approach for 3D lane detection Introduction This is a pytorch implementation of Gen-LaneNet, which p

Yuliang Guo 233 Jan 6, 2023
A Data Annotation Tool for Semantic Segmentation, Object Detection and Lane Line Detection.(In Development Stage)

Data-Annotation-Tool How to Run this Tool? To run this software, follow the steps: git clone https://github.com/Autonomous-Car-Project/Data-Annotation

TiVRA AI 13 Aug 18, 2022
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

eXtreme Gradient Boosting Community | Documentation | Resources | Contributors | Release Notes XGBoost is an optimized distributed gradient boosting l

Distributed (Deep) Machine Learning Community 23.6k Dec 31, 2022
High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

What is xLearn? xLearn is a high performance, easy-to-use, and scalable machine learning package that contains linear model (LR), factorization machin

Chao Ma 3k Jan 3, 2023
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

eXtreme Gradient Boosting Community | Documentation | Resources | Contributors | Release Notes XGBoost is an optimized distributed gradient boosting l

Distributed (Deep) Machine Learning Community 20.6k Feb 13, 2021
High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

What is xLearn? xLearn is a high performance, easy-to-use, and scalable machine learning package that contains linear model (LR), factorization machin

Chao Ma 2.8k Feb 12, 2021
A web-based application for quick, scalable, and automated hyperparameter tuning and stacked ensembling in Python.

Xcessiv Xcessiv is a tool to help you create the biggest, craziest, and most excessive stacked ensembles you can think of. Stacked ensembles are simpl

Reiichiro Nakano 1.3k Nov 17, 2022
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 6.9k Jan 4, 2023
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 5.7k Feb 12, 2021
Fast and scalable uncertainty quantification for neural molecular property prediction, accelerated optimization, and guided virtual screening.

Evidential Deep Learning for Guided Molecular Property Prediction and Discovery Ava Soleimany*, Alexander Amini*, Samuel Goldman*, Daniela Rus, Sangee

Alexander Amini 75 Dec 15, 2022