Locally Differentially Private Distributed Deep Learning via Knowledge Distillation (LDP-DL)

Di Zhuang

Last update: Jan 8, 2023

Related tags

Deep Learning LDP-DL

Overview

Locally Differentially Private Distributed Deep Learning via Knowledge Distillation (LDP-DL)

A preprint version of our paper: Link here
This is a sample code of our LDP-DL paper, including all the necessary implementations: 1) DataPartation.py: split the data for data owners and data user; 2) TeacherModel.py: train the teacher models using data owners' private data; 3) StudentModel_AQS.py: train the student model via active query sampling and knowledge distillation using data user's public data.
The splited data and trained teacher models are also provided. Due to the limitation of capacity, please download from this link:

Requirements

Pytorch 1.4.0, numpy 1.18.1, scipy 1.4.1, cuda 10.0, joblib 0.14.1

Quick Start (only tested using PyCharm on Ubuntu, Python 3.7)

Download or clone the whole repository.
Import our code as a project into PyCharm.
Run DataPartation.py to split the data.
Run TeacherModel.py to train teacher models.
Run StudentModle_AQS.py to train and evaluate student model.

Contact info: [email protected], [email protected]

You might also like...

PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation

PocketNet This is the official repository of the paper: PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and M

40 Dec 22, 2022

The official implementation of CVPR 2021 Paper: Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation.

Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation This repository is the official implementation of CVPR 2021 paper:

9 Nov 14, 2022

Light-weight network, depth estimation, knowledge distillation, real-time depth estimation, auxiliary data.

light-weight-depth-estimation Boosting Light-Weight Depth Estimation Via Knowledge Distillation, https://arxiv.org/abs/2105.06143 Junjie Hu, Chenyou F

13 Dec 10, 2022

[NeurIPS-2021] Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data

MosaicKD Code for NeurIPS-21 paper "Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data" 1. Motivation Natural images share common l

37 Nov 10, 2022

Instance-conditional Knowledge Distillation for Object Detection

Instance-conditional Knowledge Distillation for Object Detection This is a MegEngine implementation of the paper "Instance-conditional Knowledge Disti

47 Nov 17, 2022

Knowledge Distillation Toolbox for Semantic Segmentation

SegDistill: Toolbox for Knowledge Distillation on Semantic Segmentation Networks This repo contains the supported code and configuration files for Seg

9 Dec 12, 2022

Focal and Global Knowledge Distillation for Detectors

FGD Paper: Focal and Global Knowledge Distillation for Detectors Install MMDetection and MS COCO2017 Our codes are based on MMDetection. Please follow

261 Dec 23, 2022

Pytorch implementation for Patient Knowledge Distillation for BERT Model Compression

Patient Knowledge Distillation for BERT Model Compression Knowledge distillation for BERT model Installation Run command below to install the environm

180 Dec 19, 2022

PyTorch implementation of paper A Fast Knowledge Distillation Framework for Visual Recognition.

FKD: A Fast Knowledge Distillation Framework for Visual Recognition Official PyTorch implementation of paper A Fast Knowledge Distillation Framework f

129 Dec 24, 2022

Locally Differentially Private Distributed Deep Learning via Knowledge Distillation (LDP-DL)

Related tags

Overview

Locally Differentially Private Distributed Deep Learning via Knowledge Distillation (LDP-DL)

Requirements

Quick Start (only tested using PyCharm on Ubuntu, Python 3.7)

You might also like...

PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation

The official implementation of CVPR 2021 Paper: Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation.

Light-weight network, depth estimation, knowledge distillation, real-time depth estimation, auxiliary data.

[NeurIPS-2021] Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data

Instance-conditional Knowledge Distillation for Object Detection

Knowledge Distillation Toolbox for Semantic Segmentation

Focal and Global Knowledge Distillation for Detectors

Pytorch implementation for Patient Knowledge Distillation for BERT Model Compression

PyTorch implementation of paper A Fast Knowledge Distillation Framework for Visual Recognition.

Owner

Di Zhuang

TF2 implementation of knowledge distillation using the "function matching" hypothesis from the paper Knowledge distillation: A good teacher is patient and consistent by Beyer et al.

Official implementation of the paper "Lightweight Deep CNN for Natural Image Matting via Similarity Preserving Knowledge Distillation"

Official implementation for (Refine Myself by Teaching Myself : Feature Refinement via Self-Knowledge Distillation, CVPR-2021)

Official implementation for (Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching, AAAI-2021)

This is the official pytorch implementation of Student Helping Teacher: Teacher Evolution via Self-Knowledge Distillation(TESKD)

Weak-supervised Visual Geo-localization via Attention-based Knowledge Distillation

Block-wisely Supervised Neural Architecture Search with Knowledge Distillation (CVPR 2020)

AMTML-KD: Adaptive Multi-teacher Multi-level Knowledge Distillation

Code implementation of Data Efficient Stagewise Knowledge Distillation paper.

TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.