Group Fisher Pruning for Practical Network Compression(ICML2021)

Last update: Dec 13, 2022

Overview

Group Fisher Pruning for Practical Network Compression (ICML2021)

By Liyang Liu*, Shilong Zhang*, Zhanghui Kuang, Jing-Hao Xue, Aojun Zhou, Xinjiang Wang, Yimin Chen, Wenming Yang, Qingmin Liao, Wayne Zhang

Updates

All one stage models of Detection has been released (21/6/2021)

NOTES

All models about detection has been released. The classification models will be released later, because we want to refactor all our code into a Hook , so that it can become a more general tool for all tasks in OpenMMLab.

We will continue to improve this method and apply it to more other tasks, such as segmentation and pose.

The layer grouping algorithm is implemtated based on the AutoGrad of Pytorch, If you are not familiar with this feature and you can read Chinese, then these materials may be helpful to you.

Introduction

1. Compare with state-of-the-arts.

2. Can be applied to various complicated structures and various tasks.

3. Boosting inference speed on GPU under same flops.

Get Started

1. Creat a basic environment with pytorch 1.3.0 and mmcv-full

Due to the frequent changes of the autograd interface, we only guarantee the code works well in `pytorch==1.3.0`.

Creat the environment

conda create -n open-mmlab python=3.7 -y
conda activate open-mmlab

Install PyTorch 1.3.0 and corresponding torchvision.

conda install pytorch=1.3.0 cudatoolkit=10.0 torchvision=0.2.2 -c pytorch

Build the mmcv-full from source with pytorch 1.3.0 and cuda 10.0

Please use gcc-5.4 and nvcc 10.0

 git clone https://github.com/open-mmlab/mmcv.git
 cd mmcv
 MMCV_WITH_OPS=1 pip install -e .

2. Install the corresponding codebase in OpenMMLab.

e.g. MMdetection

pip install mmdet==2.13.0

3. Pruning the model.

e.g. Detection

cd detection

Modify the load_from as the path to the baseline model in of xxxx_pruning.py

# for slurm train
sh tools/slurm_train.sh PATITION_NAME JOB_NAME configs/retina/retina_pruning.py work_dir
# for slurm_test
sh tools/slurm_test.sh PATITION_NAME JOB_NAME configs/retina/retina_pruning.py PATH_CKPT --eval bbox
# for torch.dist
# sh tools/dist_train.sh configs/retina/retina_pruning.py 8

4. Finetune the model.

e.g. Detection

cd detection

Modify the deploy_from as the path to the pruned model in custom_hooks of xxxx_finetune.py

# for slurm train
sh tools/slurm_train.sh PATITION_NAME JOB_NAME configs/retina/retina_finetune.py work_dir
# for slurm test
sh tools/slurm_test.sh PATITION_NAME JOB_NAME configs/retina/retina_fintune.py PATH_CKPT --eval bbox
# for torch.dist
# sh tools/dist_train.sh configs/retina/retina_finetune.py 8

Models

Detection

Method	Backbone	Baseline(mAP)	Finetuned(mAP)	Download
RetinaNet	R-50-FPN	36.5	36.5	Baseline/Pruned/Finetuned
ATSS*	R-50-FPN	38.1	37.9	Baseline/Pruned/Finetuned
PAA*	R-50-FPN	39.0	39.4	Baseline/Pruned/Finetuned
FSAF	R-50-FPN	37.4	37.4	Baseline/Pruned/Finetuned

* indicate with no Group Normalization in heads.

Classification

Coming soon.

Please cite our paper in your publications if it helps your research.

@InProceedings{liu2021group,
  title = {Group Fisher Pruning for Practical Network Compression},
  author =       {Liu, Liyang and Zhang, Shilong and Kuang, Zhanghui and Zhou, Aojun and Xue, Jing-Hao and Wang, Xinjiang and Chen, Yimin and Yang, Wenming and Liao, Qingmin and Zhang, Wayne},
  booktitle = {Proceedings of the 38th International Conference on Machine Learning},
  year = {2021},
  series = {Proceedings of Machine Learning Research},
  month = {18--24 Jul},
  publisher ={PMLR},
}

Comments

How prune the FC layer?

Hi, I tried to complete the codes of pruning faster rcnn, but I don't know how to prune the FC layer in roi_head. I have finished the twostage_wrapper function as shown below But I got error. Could you help me see where the problem is? Thanks!

opened by mls1999725 8
How compute the speedup on GPUs?

Hi, I wonder how to compute speedup on GPUs? Is it the total inference time of CPU divide by that on GPU? If so, I wonder how to calculate the total time on the CPU by mmdetection, do I need to put all the data on the CPU for retraining? Or just need to put the data on the CPU during inference? Looking forward to reply！

opened by mls1999725 4
How compute the acts of FC layer?

Hi, I'm trying to complete the pruning codes of FC layer. I wonder how to compute the acts of FC layer, is it n times oc where n is the batch size and oc is the channel numbers of output feature?

opened by mls1999725 4
The batch size and number of gpu during pruning.

Hi, I basically completed the pruning codes of faster-rcnn. I wonder the batch size and the number of gpu during pruning faster-rcnn, is it 16 on 8 2080ti which is same as all detection models? Thank you very much!

opened by mls1999725 3
An issue about running the code

I create a virtual environment, open-mmlab, with python=3.7, pytorch=1.3.0, cudatoolkit=10.0, mmcv=1.3.16, mmdet=2.13.0. However, it still can not run.

File "tools/train.py", line 16, in
from mmdet.apis import set_random_seed, train_detector ImportError: libcudart.so.10.1: cannot open shared object file: No such file or directory

opened by MaltoseFlower 2
Finetuning speed is too slow.

Hi, I'm very interested in your work. When I fintune the RetinaNet of your pruned model, the finetuning process will last for 2 days, which is very different from the 7 hours in the paper, as shown in the figure below. One 1080ti is used and I didn't use slurm. The command for finetuning I used is python -u tools/train.py configs/retina/retina_finetune.py --work-dir ./ --gpu-ids 0

opened by mls1999725 2
Some questions about the paper and code
Thanks for the great work, I have some questions about this code

In the paper you mentioned coupled channels, I understand what it means is that C2/C5 (you described in Figure 4) receive the same input. Is there any other meaning besides this?

2. For this common concat operation in yolo, can your code still handle it normally？Because the input comes from two ancestors, and it seems that the number of channels cannot correspond to the function **def construct_outchannel_masks(self)** to construct the mask， Hope to get your reply.
opened by syswyl 1
Why does the gradient multiply the input?

https://github.com/jshilong/FisherPruning/blob/b28ea9185f1269ef4597303ccb6be3300c7e1b93/detection/tools/fisher_pruning.py#L432 How it work if we have different in_channels and out_channels?

opened by yumath 0
Resume pruning

Hi, thanks for this awesome work. I'm wondering if there is an option to continue pruning, I tried setting pruning=True and deploy_from=checkpoint to achieve it, but it didn't work. And then I modify the code in https://github.com/jshilong/FisherPruning/blob/b28ea9185f1269ef4597303ccb6be3300c7e1b93/detection/tools/fisher_pruning.py#L113, making model loading mask and checkpoint when pruning. At the same time I comment the code in https://github.com/jshilong/FisherPruning/blob/b28ea9185f1269ef4597303ccb6be3300c7e1b93/detection/tools/fisher_pruning.py#L133 to prevent model sets mask to 1. Although it can obtain information about the pruned channels, it still prunes them according to the initial model.

Is there anything else I need to change?

opened by Janspiry 0
The batch size and number of gpu during pruning.

Hi, I basically completed the pruning codes of faster-rcnn. I wonder the batch size and the number of gpu during pruning faster-rcnn, is it 16 on 8 2080ti which is same as all detection models? Thank you very much!

opened by mls1999725 0
The batch size and number of gpu during pruning.

Hi, I basically completed the pruning codes of faster-rcnn. I wonder the batch size and the number of gpu during pruning faster-rcnn, is it 16 on 8 2080ti which is same as all detection models? Thank you very much!

opened by mls1999725 0
Question on Algorithm 1 (Layer Grouping)

In ResNeXt blocks, conv2 is a GConv and should be grouped with its child conv3. However, if conv2 is not in any group while we assign a group for conv3, e.g. conv2 is behind conv3 in \mathbb{L}, conv3 will not be assigned to the same group with conv2. Instead, it will be assigned to a new group. The conv2 will not be assigned to this group since conv2 and conv3 have no shared parent. When we modify the make_groups function according to this algorithm, it indeed fails to group the conv2&conv3 in ResNeXt50 blocks though the found ancestors are correct.

Could u please help check this problem?

Now in this repo, we tentatively change this function to and it works.

opened by Ben-Louis 2
The finetuned ATSS model file is bad.

Hi, the given finetuned ATSS model can't be tested, the error is as follows, maybe the file is incomplete. And I have tested finetuned models of RetinaNet and PAA, they are normal.
bug

opened by mls1999725 2

Owner

Shilong Zhang

GitHub

A Closer Look at Structured Pruning for Neural Network Compression

A Closer Look at Structured Pruning for Neural Network Compression Code used to reproduce experiments in https://arxiv.org/abs/1810.04622. To prune, w

140 Dec 5, 2022

Official implementation of the ICML2021 paper "Elastic Graph Neural Networks"

ElasticGNN This repository includes the official implementation of ElasticGNN in the paper "Elastic Graph Neural Networks" [ICML 2021]. Xiaorui Liu, W

34 Dec 4, 2022

YOLOv5 Series Multi-backbone, Pruning and quantization Compression Tool Box.

YOLOv5-Compression Update News Requirements 环境安装 pip install -r requirements.txt Evaluation metric Visdrone Model mAP mAP@50 Parameters(M) GFLOPs FPS@

719 Jan 2, 2023

Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme (NeurIPS2021)

Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme (NeurIPS2021) Overview Prerequisites Linux Pytho

34 Mar 31, 2022

An Image compression simulator that uses Source Extractor and Monte Carlo methods to examine the post compressive effects different compression algorithms have.

ImageCompressionSimulation An Image compression simulator that uses Source Extractor and Monte Carlo methods to examine the post compressive effects o

1 Dec 11, 2021

This code reproduces the results of the paper, "Measuring Data Leakage in Machine-Learning Models with Fisher Information"

Fisher Information Loss This repository contains code that can be used to reproduce the experimental results presented in the paper: Awni Hannun, Chua

43 Dec 30, 2022

Group Fisher Pruning for Practical Network Compression(ICML2021)

Related tags

Overview

Updates

NOTES

Introduction

1. Compare with state-of-the-arts.

2. Can be applied to various complicated structures and various tasks.

3. Boosting inference speed on GPU under same flops.

Get Started

1. Creat a basic environment with pytorch 1.3.0 and mmcv-full

Due to the frequent changes of the autograd interface, we only guarantee the code works well in pytorch==1.3.0.

Please use gcc-5.4 and nvcc 10.0

2. Install the corresponding codebase in OpenMMLab.

3. Pruning the model.

4. Finetune the model.

Models

Detection

Classification

Please cite our paper in your publications if it helps your research.

Comments

Owner

Shilong Zhang

A Closer Look at Structured Pruning for Neural Network Compression

Official implementation of the ICML2021 paper "Elastic Graph Neural Networks"

YOLOv5 Series Multi-backbone, Pruning and quantization Compression Tool Box.

Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme (NeurIPS2021)

An Image compression simulator that uses Source Extractor and Monte Carlo methods to examine the post compressive effects different compression algorithms have.

This code reproduces the results of the paper, "Measuring Data Leakage in Machine-Learning Models with Fisher Information"

Official PyTorch code for CVPR 2020 paper "Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision"

A curated list of neural network pruning resources.

Code for PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning

Network Pruning That Matters: A Case Study on Retraining Variants (ICLR 2021)

Ensemble Knowledge Guided Sub-network Search and Fine-tuning for Filter Pruning

Code for paper: Group-CAM: Group Score-Weighted Visual Explanations for Deep Convolutional Networks

BC3407-Group-5-Project - BC3407 Group Project With Python

Code of paper "CDFI: Compression-Driven Network Design for Frame Interpolation", CVPR 2021

Block Sparse movement pruning

This repo contains the official implementations of EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis

Learned Token Pruning for Transformers

Learnable Motion Coherence for Correspondence Pruning

A Python implementation of the Locality Preserving Matching (LPM) method for pruning outliers in image matching.

Due to the frequent changes of the autograd interface, we only guarantee the code works well in `pytorch==1.3.0`.