Few-shot Neural Architecture Search

Related tags

Deep Learning few-shot-NAS
Overview

Few-shot Neural Architecture Search

@misc{zhao2020fewshot,
      title={Few-shot Neural Architecture Search}, 
      author={Yiyang Zhao and Linnan Wang and Yuandong Tian and Rodrigo Fonseca and Tian Guo},
      year={2020},
      eprint={2006.06863},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Introduction

One-shot Neural Architecture Search uses a single supernet to approximate the performance each architecture. However, this performance estimation is super inaccurate because of co-adaption among operations in supernet. Few-shot NAS uses multiple supernets with less edges(operations) and each of them covers different regions of the search space to alleviate the undesired co-adaption. Compared to one-shot NAS, few-shot NAS greatly improve the performance of architecture evaluation with a small increase of overhead. Please click here to see our paper.

Few-shot NAS on NasBench201

Please refer here to see how to use few-shot NAS improve the search performance on NasBench201.

Few-shot NAS on Cifar10

Please refer here to test our state-of-the-art models searched by few-shot NAS.

You might also like...
Deep Multimodal Neural Architecture Search
Deep Multimodal Neural Architecture Search

MMNas: Deep Multimodal Neural Architecture Search This repository corresponds to the PyTorch implementation of the MMnas for visual question answering

Vision and Language Group@ MIL 23 Dec 21, 2022
Official implementation of Rethinking Graph Neural Architecture Search from Message-passing (CVPR2021)
Official implementation of Rethinking Graph Neural Architecture Search from Message-passing (CVPR2021)

Rethinking Graph Neural Architecture Search from Message-passing Intro The GNAS can automatically learn better architecture with the optimal depth of

Shaofei Cai 48 Sep 30, 2022
Block-wisely Supervised Neural Architecture Search with Knowledge Distillation (CVPR 2020)
Block-wisely Supervised Neural Architecture Search with Knowledge Distillation (CVPR 2020)

DNA This repository provides the code of our paper: Blockwisely Supervised Neural Architecture Search with Knowledge Distillation. Illustration of DNA

Changlin Li 215 Dec 19, 2022
"NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search".

NAS-Bench-301 This repository containts code for the paper: "NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search". The

AutoML-Freiburg-Hannover 57 Nov 30, 2022
Code release to accompany paper "Geometry-Aware Gradient Algorithms for Neural Architecture Search."

Geometry-Aware Gradient Algorithms for Neural Architecture Search This repository contains the code required to run the experiments for the DARTS sear

null 18 May 27, 2022
Official PyTorch implementation of
Official PyTorch implementation of "Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets" (ICLR 2021)

Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets This is the official PyTorch implementation for the paper Rapid Neural A

null 48 Dec 26, 2022
code for "AttentiveNAS Improving Neural Architecture Search via Attentive Sampling"

code for "AttentiveNAS Improving Neural Architecture Search via Attentive Sampling"

Facebook Research 94 Oct 26, 2022
PyTorch implementation of
PyTorch implementation of "Efficient Neural Architecture Search via Parameters Sharing"

Efficient Neural Architecture Search (ENAS) in PyTorch PyTorch implementation of Efficient Neural Architecture Search via Parameters Sharing. ENAS red

Taehoon Kim 2.6k Dec 31, 2022
An implementation for Neural Architecture Search with Random Labels (CVPR 2021 poster) on Pytorch.
An implementation for Neural Architecture Search with Random Labels (CVPR 2021 poster) on Pytorch.

Neural Architecture Search with Random Labels(RLNAS) Introduction This project provides an implementation for Neural Architecture Search with Random L

null 18 Nov 8, 2022
Comments
  • Cannot reproduce Cifar10 code

    Cannot reproduce Cifar10 code

    Hi I cannot reproduce cifar10 code according to the instructions: i get this error

    /few-shot-NAS/Few-Shot-NAS_cifar10$ python test.py --arch=small --gpu=1
12/07 11:31:05 PM gpu device = 1
12/07 11:31:05 PM args = Namespace(arch='small', auxiliary_weight=0.4, batch_size=96, cutout=False, cutout_length=16, data='../data', drop_path_prob=0.2, epochs=600, gpu=1, layers=20, lr=0.025, model_path='saved_models', momentum=0.9, report_freq=50, save='EXP', seed=0, wd=0.0003)
12/07 11:31:06 PM param size = 3.789982MB
Files already downloaded and verified
Traceback (most recent call last):
  File "test.py", line 134, in <module>
    main()
  File "test.py", line 97, in main
    valid_acc, valid_obj = infer(valid_queue, model, criterion)
  File "test.py", line 117, in infer
    prec1, prec5 = utils.accuracy(logits, target, topk=(1, 5))
  File "/home/few-shot-NAS/Few-Shot-NAS_cifar10/utils.py", line 43, in accuracy
    correct_k = correct[:k].view(-1).float().sum(0)
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

    Torch:1.11.5 numpy:1.19

    opened by prabhant 1
  • How did you split the search space of OFA into 5 sub spaces?

    How did you split the search space of OFA into 5 sub spaces?

    I see that "We split 5 sub-supernets" in the SI. I can't come up with a way to split the search space into 5 disjoint partitions. And how did you train the sub-supernets? OFA is trained with progressive shrink. Did you still use progressive shrink when training sub supernets?

    opened by peterwu4084 0
  • reproduce your one/few-shot results on nas-bench 201

    reproduce your one/few-shot results on nas-bench 201

    I try to reproduce your one-shpt results on nas-bench 201. By running the code https://github.com/aoiang/few-shot-NAS/blob/main/Few-Shot_NasBench201/supernet/one-shot/run.sh

    and eval https://github.com/aoiang/few-shot-NAS/blob/main/Few-Shot_NasBench201/supernet/one-shot/eval.sh

    However, your eval.sh is

    for (( c=0; c<=4; c++ )) do OMP_NUM_THREADS=4 python ./exps/supernet/one-shot-supernet_eval.py
    --save_dir ${save_dir} --max_nodes ${max_nodes} --channel ${channel} --num_cells ${num_cells}
    --dataset ${dataset} --data_path ${data_path}
    --search_space_name ${space}
    --arch_nas_dataset ${benchmark_file}
    --config_path configs/nas-benchmark/algos/FEW-SHOT-SUPERNET.config
    --track_running_stats ${BN}
    --select_num 100
    --output_dir ${OUTPUT}
    --workers 4 --print_freq 200 --rand_seed 0 --edge_op ${c} done

    Why using te FEW-SHOT-SUPERNET.config?

    opened by Jxu-Thu 23
Owner
Yiyang Zhao
Yiyang Zhao
GitHub
[CVPR21] LightTrack: Finding Lightweight Neural Network for Object Tracking via One-Shot Architecture Search

LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search The official implementation of the paper LightTra

Multimedia Research 290 Dec 24, 2022
Densely Connected Search Space for More Flexible Neural Architecture Search (CVPR2020)

DenseNAS The code of the CVPR2020 paper Densely Connected Search Space for More Flexible Neural Architecture Search. Neural architecture search (NAS)

Jamin Fong 291 Nov 18, 2022
Few-NERD: Not Only a Few-shot NER Dataset

Few-NERD: Not Only a Few-shot NER Dataset This is the source code of the ACL-IJCNLP 2021 paper: Few-NERD: A Few-shot Named Entity Recognition Dataset.

THUNLP 319 Dec 30, 2022
Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"

T-Few This repository contains the official code for the paper: "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learni

null 220 Dec 31, 2022
DeepHyper: Scalable Asynchronous Neural Architecture and Hyperparameter Search for Deep Neural Networks

What is DeepHyper? DeepHyper is a software package that uses learning, optimization, and parallel computing to automate the design and development of

DeepHyper Team 214 Jan 8, 2023
Model search is a framework that implements AutoML algorithms for model architecture search at scale

Model search (MS) is a framework that implements AutoML algorithms for model architecture search at scale. It aims to help researchers speed up their exploration process for finding the right model architecture for their classification problems (i.e., DNNs with different types of layers).

Google 3.2k Dec 31, 2022
Deep Image Search is an AI-based image search engine that includes deep transfor learning features Extraction and tree-based vectorized search.

Deep Image Search - AI-Based Image Search Engine Deep Image Search is an AI-based image search engine that includes deep transfer learning features Ex

null 139 Jan 1, 2023
[ICLR 2021] "Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective" by Wuyang Chen, Xinyu Gong, Zhangyang Wang

Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective [PDF] Wuyang Chen, Xinyu Gong, Zhangyang Wang In ICLR 2

VITA 156 Nov 28, 2022
[ICLR 2021] HW-NAS-Bench: Hardware-Aware Neural Architecture Search Benchmark

HW-NAS-Bench: Hardware-Aware Neural Architecture Search Benchmark Accepted as a spotlight paper at ICLR 2021. Table of content File structure Prerequi

null 72 Jan 3, 2023
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search

BossNAS This repository contains PyTorch evaluation code, retraining code and pretrained models of our paper: BossNAS: Exploring Hybrid CNN-transforme

Changlin Li 127 Dec 26, 2022