Using Self-Supervised Pretext Tasks for Active Learning - Official Pytorch Implementation

Related tags

Deep Learning PT4AL
Overview

Using Self-Supervised Pretext Tasks for Active Learning - Official Pytorch Implementation

Experiment Setting:

  • CIFAR10 (downloaded and saved in ./DATA
  • Rotation prediction for pretext task

Prerequisites:

Python >= 3.7

CUDA = 11.0

PyTorch = 1.7.1

numpy >= 1.16.0

Running the Code

To train the rotation predition task on the unlabeled set:

python rotation.py

To extract pretext task losses and create batches:

python make_batches.py

To evaluate on active learning task:

python main.py
Comments
  • About Test Dataset

    About Test Dataset

    Hi, I have a question about test in "main.py".

    As I read your code, you use TRAIN dataset for test section(in main.py, line 44, 45) Is there any reasons why you've done that?

    thanks a lot in advance.

    opened by ia-gu 6
  • Your code use trainset for test phase ?

    Your code use trainset for test phase ?

    image image

    In main.py, your testset looks like above: is_train=False, path=None. So, self.img_path in Loader2 will take data from glob.glob('./DATA/train//'). Which means, you are using traindataset for testing ?

    opened by quannguyen268 1
  • Question abou main.py testset

    Question abou main.py testset

    Hi, thank you so much for your sharing your work.

    main.py 44 line testset = Loader2(is_train=False, transform=transform_test)

    I think it works trainset, so should I fix it like this? testset = Loader(is_train=False, transform=transform_test)

    opened by lbg030 1
  • Training Sorted batches for AL cycle

    Training Sorted batches for AL cycle

    Hi, thank you so much for your sharing your work.

    So I wanted to ask one thing once we are done with the pretext task and sorting with batches, we will train for an active learning cycle from scratch?

    There is no information from the previous training to the next training which is Active Learning training. Am I getting correct?

    opened by manza-ari 1
  • Extending current work

    Extending current work

    @johnsk95 thanks for sharing the code base, i have a query can we use the existing source code to extend it to task like object detection and segmentation? if so what changes have to be made Thanks in advance

    opened by abhigoku10 1
  • Can not reproduce

    Can not reproduce

    "The current reproduction result is the performance of 91 to 93" I can not reproduce above accuracy. I tried to reproduce three times. The final results that I got are 89.45, 89.71, 90.05

    opened by thanhnt-2658 1
  • Why normalize the output before calculate softmax

    Why normalize the output before calculate softmax

    I don't understand why it is needed to run the below line. I tried to reproduce your results. In my experiments, without this line, I even achieved better results. https://github.com/johnsk95/PT4AL/blob/main/main.py#L171

    opened by thanhnt-2658 0
  • The Imagenet-67 dataset

    The Imagenet-67 dataset

    Hi,

    Thank you for the great work! I'm just wondering if by any chance you could release the mapping of the 67 superclasses of Imagenet you used? Eg, the mapping of each superclass to its corresponding Imagenet class (wordnet id if possible).

    opened by JieyuZ2 0
  • 'stty' is not recognized as an internal or external command, operable program or batch file.

    'stty' is not recognized as an internal or external command, operable program or batch file.

    I got this error when running the rotation.py

    full traceback error: 'stty' is not recognized as an internal or external command, operable program or batch file. Traceback (most recent call last): File "", line 1, in File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\multiprocessing\spawn.py", line 105, in spawn_main exitcode = _main(fd) File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\multiprocessing\spawn.py", line 114, in _main prepare(preparation_data) File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\multiprocessing\spawn.py", line 225, in prepare _fixup_main_from_path(data['init_main_from_path']) File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path run_name="mp_main") File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\runpy.py", line 85, in _run_code exec(code, run_globals) File "D:\Omid\UPB\SVM\AL\PT4AL-main\rotation.py", line 159, in train(epoch) File "D:\Omid\UPB\SVM\AL\PT4AL-main\rotation.py", line 76, in train for batch_idx, (inputs, inputs1, inputs2, inputs3, targets, targets1, targets2, targets3) in enumerate(trainloader): File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 355, in iter return self._get_iterator() File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 301, in _get_iterator return _MultiProcessingDataLoaderIter(self) File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 914, in init w.start() File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\multiprocessing\process.py", line 105, in start self._popen = self._Popen(self) File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj) File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\multiprocessing\popen_spawn_win32.py", line 33, in init prep_data = spawn.get_preparation_data(process_obj._name) File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\multiprocessing\spawn.py", line 143, in get_preparation_data _check_not_importing_main() File "C:\Users\CEOSpaceTech\Miniconda3\envs\env_pytorch\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main is not going to be frozen to produce an executable.''') RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:
    
            if __name__ == '__main__':
                freeze_support()
                ...
    
        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.
    
    opened by omid-ghozatlou 0
Owner
John Seon Keun Yi
MSCS Georgia Tech
John Seon Keun Yi
Official code for the paper "Why Do Self-Supervised Models Transfer? Investigating the Impact of Invariance on Downstream Tasks".

Why Do Self-Supervised Models Transfer? Investigating the Impact of Invariance on Downstream Tasks This repository contains the official code for the

Linus Ericsson 11 Dec 16, 2022
gym-anm is a framework for designing reinforcement learning (RL) environments that model Active Network Management (ANM) tasks in electricity distribution networks.

gym-anm is a framework for designing reinforcement learning (RL) environments that model Active Network Management (ANM) tasks in electricity distribution networks. It is built on top of the OpenAI Gym toolkit.

Robin Henry 99 Dec 12, 2022
Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

null 49 Nov 23, 2022
[BMVC 2021] Official PyTorch Implementation of Self-supervised learning of Image Scale and Orientation Estimation

Self-Supervised Learning of Image Scale and Orientation Estimation (BMVC 2021) This is the official implementation of the paper "Self-Supervised Learn

Jongmin Lee 17 Nov 10, 2022
An official PyTorch implementation of the TKDE paper "Self-Supervised Graph Representation Learning via Topology Transformations".

Self-Supervised Graph Representation Learning via Topology Transformations This repository is the official PyTorch implementation of the following pap

Hsiang Gao 2 Oct 31, 2022
Official PyTorch implementation of "Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Recognition" in AAAI2022.

AimCLR This is an official PyTorch implementation of "Contrastive Learning from Extremely Augmented Skeleton Sequences for Self-supervised Action Reco

Gty 44 Dec 17, 2022
An official PyTorch Implementation of Boundary-aware Self-supervised Learning for Video Scene Segmentation (BaSSL)

An official PyTorch Implementation of Boundary-aware Self-supervised Learning for Video Scene Segmentation (BaSSL)

Kakao Brain 72 Dec 28, 2022
Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR

UniSpeech The family of UniSpeech: UniSpeech (ICML 2021): Unified Pre-training for Self-Supervised Learning and Supervised Learning for ASR UniSpeech-

Microsoft 282 Jan 9, 2023
Repository providing a wide range of self-supervised pretrained models for computer vision tasks.

Hierarchical Pretraining: Research Repository This is a research repository for reproducing the results from the project "Self-supervised pretraining

Colorado Reed 53 Nov 9, 2022
UniLM AI - Large-scale Self-supervised Pre-training across Tasks, Languages, and Modalities

Pre-trained (foundation) models across tasks (understanding, generation and translation), languages (100+ languages), and modalities (language, image, audio, vision + language, audio + language, etc.)

Microsoft 7.6k Jan 1, 2023
Official PyTorch implementation and pretrained models of the paper Self-Supervised Classification Network

Self-Classifier: Self-Supervised Classification Network Official PyTorch implementation and pretrained models of the paper Self-Supervised Classificat

Elad Amrani 24 Dec 21, 2022
deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.

deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.

null 63 Oct 17, 2022
This is an official implementation for "Self-Supervised Learning with Swin Transformers".

Self-Supervised Learning with Vision Transformers By Zhenda Xie*, Yutong Lin*, Zhuliang Yao, Zheng Zhang, Qi Dai, Yue Cao and Han Hu This repo is the

Swin Transformer 529 Jan 2, 2023
The official implementation of the paper, "SubTab: Subsetting Features of Tabular Data for Self-Supervised Representation Learning"

SubTab: Author: Talip Ucar ([email protected]) The official implementation of the paper, SubTab: Subsetting Features of Tabular Data for Self-Supervis

AstraZeneca 98 Dec 29, 2022
Official implementation of "Motif-based Graph Self-Supervised Learning forMolecular Property Prediction"

Motif-based Graph Self-Supervised Learning for Molecular Property Prediction Official Pytorch implementation of NeurIPS'21 paper "Motif-based Graph Se

zaixi 71 Dec 20, 2022
Official implementation of the method ContIG, for self-supervised learning from medical imaging with genomics

ContIG: Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics This is the code implementation of the paper "ContIG: Self-s

Digital Health & Machine Learning 22 Dec 13, 2022
PyTorch implementation of "Contrast to Divide: self-supervised pre-training for learning with noisy labels"

Contrast to Divide: self-supervised pre-training for learning with noisy labels This is an official implementation of "Contrast to Divide: self-superv

null 55 Nov 23, 2022
A PyTorch implementation of "Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning", IJCAI-21

MERIT A PyTorch implementation of our IJCAI-21 paper Multi-Scale Contrastive Siamese Networks for Self-Supervised Graph Representation Learning. Depen

Graph Analysis & Deep Learning Laboratory, GRAND 32 Jan 2, 2023
Usable Implementation of "Bootstrap Your Own Latent" self-supervised learning, from Deepmind, in Pytorch

Bootstrap Your Own Latent (BYOL), in Pytorch Practical implementation of an astoundingly simple method for self-supervised learning that achieves a ne

Phil Wang 1.4k Dec 29, 2022