SWAD: Domain Generalization by Seeking Flat Minima (NeurIPS'21)
Official PyTorch implementation of SWAD: Domain Generalization by Seeking Flat Minima.
Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, Sungrae Park.
Note that this project is built upon DomainBed@3fe9d7.
Preparation
Dependencies
pip install -r requirements.txt
Datasets
python -m domainbed.scripts.download --data_dir=/my/datasets/path
Environments
Environment details used for our study.
Python: 3.8.6
PyTorch: 1.7.0+cu92
Torchvision: 0.8.1+cu92
CUDA: 9.2
CUDNN: 7603
NumPy: 1.19.4
PIL: 8.0.1
How to Run
train_all.py script conducts multiple leave-one-out cross-validations for all target domain.
python train_all.py exp_name --dataset PACS --data_dir /my/datasets/path
Experiment results are reported as a table. In the table, the row SWAD indicates out-of-domain accuracy from SWAD. The row SWAD (inD) indicates in-domain validation accuracy.
Example results:
+------------+--------------+---------+---------+---------+---------+
| Selection | art_painting | cartoon | photo | sketch | Avg. |
+------------+--------------+---------+---------+---------+---------+
| oracle | 82.245% | 85.661% | 97.530% | 83.461% | 87.224% |
| iid | 87.919% | 78.891% | 96.482% | 78.435% | 85.432% |
| last | 82.306% | 81.823% | 95.135% | 82.061% | 85.331% |
| last (inD) | 95.807% | 95.291% | 96.306% | 95.477% | 95.720% |
| iid (inD) | 97.275% | 96.619% | 96.696% | 97.253% | 96.961% |
| SWAD | 89.750% | 82.942% | 97.979% | 81.870% | 88.135% |
| SWAD (inD) | 97.713% | 97.649% | 97.316% | 98.074% | 97.688% |
+------------+--------------+---------+---------+---------+---------+
In this example, the DG performance of SWAD for PACS dataset is 88.135%.
If you set indomain_test option to True, the validation set is splitted to validation and test sets, and the (inD) keys become to indicate in-domain test accuracy.
Reproduce the results of the paper
We provide the instructions to reproduce the main results of the paper, Table 1 and 2. Note that the difference in a detailed environment or uncontrolled randomness may bring a little different result from the paper.
- PACS
python train_all.py PACS0 --dataset PACS --deterministic --trial_seed 0 --checkpoint_freq 100 --data_dir /my/datasets/path
python train_all.py PACS1 --dataset PACS --deterministic --trial_seed 1 --checkpoint_freq 100 --data_dir /my/datasets/path
python train_all.py PACS2 --dataset PACS --deterministic --trial_seed 2 --checkpoint_freq 100 --data_dir /my/datasets/path
- VLCS
python train_all.py VLCS0 --dataset VLCS --deterministic --trial_seed 0 --checkpoint_freq 50 --tolerance_ratio 0.2 --data_dir /my/datasets/path
python train_all.py VLCS1 --dataset VLCS --deterministic --trial_seed 1 --checkpoint_freq 50 --tolerance_ratio 0.2 --data_dir /my/datasets/path
python train_all.py VLCS2 --dataset VLCS --deterministic --trial_seed 2 --checkpoint_freq 50 --tolerance_ratio 0.2 --data_dir /my/datasets/path
- OfficeHome
python train_all.py OH0 --dataset OfficeHome --deterministic --trial_seed 0 --checkpoint_freq 100 --data_dir /my/datasets/path
python train_all.py OH1 --dataset OfficeHome --deterministic --trial_seed 1 --checkpoint_freq 100 --data_dir /my/datasets/path
python train_all.py OH2 --dataset OfficeHome --deterministic --trial_seed 2 --checkpoint_freq 100 --data_dir /my/datasets/path
- TerraIncognita
python train_all.py TR0 --dataset TerraIncognita --deterministic --trial_seed 0 --checkpoint_freq 100 --data_dir /my/datasets/path
python train_all.py TR1 --dataset TerraIncognita --deterministic --trial_seed 1 --checkpoint_freq 100 --data_dir /my/datasets/path
python train_all.py TR2 --dataset TerraIncognita --deterministic --trial_seed 2 --checkpoint_freq 100 --data_dir /my/datasets/path
- DomainNet
python train_all.py DN0 --dataset DomainNet --deterministic --trial_seed 0 --checkpoint_freq 500 --data_dir /my/datasets/path
python train_all.py DN1 --dataset DomainNet --deterministic --trial_seed 1 --checkpoint_freq 500 --data_dir /my/datasets/path
python train_all.py DN2 --dataset DomainNet --deterministic --trial_seed 2 --checkpoint_freq 500 --data_dir /my/datasets/path
Main Results
Citation
The paper will be published at NeurIPS 2021.
@inproceedings{cha2021swad,
title={SWAD: Domain Generalization by Seeking Flat Minima},
author={Cha, Junbum and Chun, Sanghyuk and Lee, Kyungjae and Cho, Han-Cheol and Park, Seunghyun and Lee, Yunsung and Park, Sungrae},
booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
year={2021}
}
License
This source code is released under the MIT license, included here.
This project includes some code from DomainBed, also MIT licensed.
Hi, I want to know if every column indicates leave-one-out cross-validations for the target domain accuracy. And oracle uses the small split of the test domain as the validation set (test-domain validation). Is this contradictory to leave-one-out?
17 Jan 1, 2023
15 Nov 21, 2022
12 Dec 7, 2022
9 Nov 3, 2021
11 Nov 27, 2022
136 Dec 21, 2022
51 Dec 3, 2022
118 Dec 25, 2022
39 Nov 10, 2022
46 Nov 9, 2022
33 Oct 12, 2022
71 Nov 18, 2022
449 Dec 27, 2022
10 Dec 2, 2022
28 Jan 8, 2023
76 Jan 3, 2023
4 Sep 13, 2022
77 Dec 16, 2022
40 Dec 20, 2022