Semi-supervised Learning for Few-shot Image-to-Image Translation

[paper]

Abstract:

In the last few years, unpaired image-to-image translation has witnessed remarkable progress. Although the latest methods are able to generate realistic images, they crucially rely on a large number of labeled images. Recently, some methods have tackled the challenging setting of few-shot image-to-image translation, reducing the labeled data requirements for the target domain during inference. In this work, we go one step further and reduce the amount of required labeled data also from the source domain during training. To do so, we propose applying semi-supervised learning via a noise-tolerant pseudo-labeling procedure. We also apply a cycle consistency constraint to further exploit the information from unlabeled images, either from the same dataset or external. Additionally, we propose several structural modifications to facilitate the image translation task under these circumstances. Our semi-supervised method for few-shot image translation, called SEMIT, achieves excellent results on four different datasets using as little as 10% of the source labels, and matches the performance of the main fully-supervised competitor using only 20% labeled data.

Overview

Dependences
Installation
Instructions
Framework
Results
References
Contact

Dependences

Python3.7, NumPy, SciPy,NVIDIA DGX1(8-V100, 32GB)
Pytorch: the version is 1.2 (https://pytorch.org/)
Dataset: Animal Face Dataset

Installation

Install pytorch

Instructions

Using 'https://github.com/yaxingwang/SEMIT.git'

You will get new folder whose name is 'SEMIT' in your current path, then use 'cd SEMIT' to enter the downloaded new folder
Download dataset or use your dataset.

Downloading the specific dataset (e.g.,'animals' ) and put target path ('datasets/animals/'). Please check 'configs/animals.yaml' to learn the path information.
Run: python train.py --config configs/animals.yaml --output_path results/animals --multigpus

Framework

Results

References

Our code heavily rely on the following projects:

[1] 'Few-Shot Unsupervised Image-to-Image Translation' by Liu et. al, code
[2] 'Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks With Octave Convolution' by Chen et. al, code

It would be helpful to understand this project if you are familiar with the above projects.

Contact

If you run into any problems with this code, please submit a bug report on the Github site of the project. For another inquries pleace contact with me: [email protected]

Comments

Question about the code

Hi, yaxingwang. Thanks for the interesting work. I have a question about the code.

Before the question, I explain my understanding.

I read your paper then I understood that SEMIT seems to require two stages to train under a semi-supervised setting -- 1. Training an auxiliary classifier with NTPL (100). 2. Training the model (translation model) with the labeled and pseudo-labeled samples.

However, I can not find the source code of the NTPL architecture and pseudo-labeling part (F_theta and M_psi). And I also checked if the file (animals_list_train.txt) contains the pseudo-labels, however, it contains the ground-truth labels. After checking the codes and appendices, I found that the hyperparameters of the classifier loss are selected as [23].

Then, should I use the classification codes from PENCIL GITHUB to get the pseudo-labels? And, is the current form of the code in this repository for the supervised setting (100% of samples are labeled) ?

Is it right that the training requires three steps? -- 1. Train the aux. classifier with labeled samples only. 2. Correct the noisy labels with PENCIL and get pseudo-labels(classification loss, correction 100 times). 3. Train SEMIT with the labels and pseudo-labels.

Thanks.

opened by FriedRonaldo 0