Naszilla is a Python library for neural architecture search (NAS)

Related tags

Deep Learning deep-learning tensorflow pytorch convolutional-networks convolutional-neural-networks cifar10 automl bananas neural-architecture-search meta-neural-network nas-algorithms
Overview

License

A repository to compare many popular NAS algorithms seamlessly across three popular benchmarks (NASBench 101, 201, and 301). You can implement your own NAS algorithm, and then easily compare it with eleven algorithms across three benchmarks.

This repository contains the official code for the following three papers:

Paper README Blog Post
A Study on Encodings for Neural Architecture Search encodings.md Blog Post
BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search bananas.md Blog Post
Exploring the Loss Landscape in Neural Architecture Search local_search.md Blog Post

Installation

Clone this repository and install its requirements (which includes nasbench, nas-bench-201, and nasbench301). It may take a few minutes.

git clone https://github.com/naszilla/naszilla
cd naszilla
cat requirements.txt | xargs -n 1 -L 1 pip install
pip install -e .

You might need to replace line 32 of src/nasbench301/surrogate_models/surrogate_models.py with a new path to the configspace file:

self.config_loader = utils.ConfigLoader(os.path.expanduser('~/naszilla/src/nasbench301/configspace.json'))

Next, download the nas benchmark datasets (either with the terminal commands below, or from their respective websites (nasbench, nas-bench-201, and nasbench301). The versions recommended for use with naszilla are nasbench_only108.tfrecord, NAS-Bench-201-v1_0-e61699.pth, and nasbench301_models_v0.9.zip. If you use a different version, you might need to edit some of the naszilla code.

# these files are 0.5GB, 2.1GB, and 1.6GB, respectively
wget https://storage.googleapis.com/nasbench/nasbench_only108.tfrecord
wget https://ndownloader.figshare.com/files/25506206?private_link=7d47bf57803227af4909 -O NAS-Bench-201-v1_0-e61699.pth
wget https://ndownloader.figshare.com/files/24693026 -O nasbench301_models_v0.9.zip
unzip nasbench301_models_v0.9.zip

Place the three downloaded benchmark data files in ~/nas_benchmark_datasets (or choose another directory and edit line 15 of naszilla/nas_benchmarks.py accordingly).

Now you have successfully installed all of the requirements to run eleven NAS algorithms on three benchmark search spaces!

Test Installation

You can test the installation by running these commands:

cd naszilla
python naszilla/run_experiments.py --search_space nasbench_101 --algo_params all_algos --queries 30 --trials 1
python naszilla/run_experiments.py --search_space nasbench_201 --algo_params all_algos --queries 30 --trials 1
python naszilla/run_experiments.py --search_space nasbench_301 --algo_params all_algos --queries 30 --trials 1

These experiments should finish running within a few minutes.

Run NAS experiments on NASBench-101/201/301 search spaces

cd naszilla
python naszilla/run_experiments.py --search_space nasbench_201 --dataset cifar100 --queries 100 --trials 100

This will test several NAS algorithms against each other on the NASBench-201 search space. Note that NASBench-201 allows you to specify one of three datasets: cifar10, cifar100, or imagenet. To customize your experiment, open naszilla/params.py. Here, you can change the algorithms and their hyperparameters. For details on running specific methods, see these docs.

Contributions

Contributions are welcome!

Reproducibility

If you have any questions about reproducing an experiment, please open an issue or email [email protected].

Citation

Please cite our papers if you use code from this repo:

@inproceedings{white2020study,
  title={A Study on Encodings for Neural Architecture Search},
  author={White, Colin and Neiswanger, Willie and Nolen, Sam and Savani, Yash},
  booktitle={Advances in Neural Information Processing Systems},
  year={2020}
}

@inproceedings{white2021bananas,
  title={BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search},
  author={White, Colin and Neiswanger, Willie and Savani, Yash},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  year={2021}
}

@inproceedings{white2021exploring,
  title={Exploring the Loss Landscape in Neural Architecture Search},
  author={White, Colin and Nolen, Sam and Savani, Yash},
  booktitle={Uncertainty in Artificial Intelligence},
  organization={PMLR},
  year={2021}
}

Contents

This repo contains encodings for neural architecture search, a variety of NAS methods (including BANANAS, a neural predictor Bayesian optimization method, and local search for NAS), and an easy interface for using multiple NAS benchmarks.

Encodings:

encodings

BANANAS:

adj_train adj_test path_train path_test

Local search:

local_search

Comments
  • The search speed of bananas algorithm compare with random is quite solow?

    The search speed of bananas algorithm compare with random is quite solow?

    The paper said We note that a plot with respect to wall-clock time would look nearly identical, since all NAS algorithms finish the set of 150 queries in roughly the same amount of time. The fastest, random search, takes 46.5 TPU hours, while BANANAS, the slowest, takes 47.0 TPU hours. I am running the code run_experiments_sequential.py in this repository. The walltimes of 150 queries with algorithms bananas, random, evolution is quite different. The bananas algorithms is ten times slower than random. The following line is the output: [359.68584871292114, 35.89821910858154, 35.73087739944458] Does I made any mistakes?

    opened by auroua 4
  • The bananas method will consume all of the system memory

    The bananas method will consume all of the system memory

    The bananas method in nas_algorithms.py will consume all of the system memory and cause memory error. I have tried on two different computer and both of the two computer get the error. Could you fix this bug?

    opened by auroua 4
  • About the Meta Neural Network training method

    About the Meta Neural Network training method

    The code chip in method banans in file nas_algorithms.py

        while query <= total_queries:
        candidates = search_space.get_candidates(data,
                                                 acq_opt_type=acq_opt_type,
                                                 encode_paths=encode_paths,
                                                 allow_isomorphisms=allow_isomorphisms,
                                                 deterministic_loss=deterministic)
        xcandidates = np.array([c[1] for c in candidates])
        predictions = []
        # train an ensemble of neural networks
        train_error = 0
        for _ in range(num_ensemble):
            meta_neuralnet = MetaNeuralnet()
            train_error += meta_neuralnet.fit(xtrain, ytrain, **metann_params)
            # predict the validation loss of the candidate architectures
            predictions.append(np.squeeze(meta_neuralnet.predict(xcandidates)))
        train_error /= num_ensemble
        if verbose:
            print('Query {}, Meta neural net train error: {}'.format(query, train_error))

    In each while loop, you created five totally new neural network and use the same training data to train the five new neural network. I can not understand why the five meta neural network is trained in this way. Using the same training data and training from scratch in each iteration.

    opened by auroua 4
  • Questions about porting algorithms to architecture search framework

    Questions about porting algorithms to architecture search framework

    Hi Colin,

    We've recently released a modular architecture search framework and we were looking at getting reference implementations of search algorithms in it. The main selling point is that we have a well-defined language to write search spaces (our paper to appear at NeurIPS 2019). Once an algorithm is implemented, it can be applied to arbitrary search spaces that a user may write. I would appreciate some help in moving your algorithms here to our framework (if your code is under the MIT license). The main questions are:

    • What search space encodings do you use?
    • How do your search algorithms plug into (i.e., interface or query) the search space?
    • What information do the search algorithms consume? Some pointers would be great!

    Our search algorithm implementation are based on the sample (get an architecture from the search space) and update (pass the results back to the searcher and update its state) API.

    Do you want to look into it? Here is the repo and colab notebook.

    Thanks, Renato

    opened by negrinho 3
  • ModuleNotFoundError: No module named 'surrogate_models'

    ModuleNotFoundError: No module named 'surrogate_models'

    Hi,

    I followed the installation steps and tried the first test python naszilla/run_experiments.py --search_space nasbench_101 --algo_params all_algos --queries 30 --trials 1. However, it resulted in ModuleNotFoundError.

    Traceback (most recent call last):
  File "naszilla/run_experiments.py", line 11, in <module>
    from naszilla.nas_benchmarks import Nasbench101, Nasbench201, Nasbench301
  File "/home/ubuntu/naszilla/naszilla/nas_benchmarks.py", line 8, in <module>
    import nasbench301 as nb
  File "/home/ubuntu/naszilla/src/nasbench301/nasbench301/__init__.py", line 1, in <module>
    from nasbench301.api import load_ensemble
  File "/home/ubuntu/naszilla/src/nasbench301/nasbench301/api.py", line 6, in <module>
    from nasbench301.surrogate_models import utils
  File "/home/ubuntu/naszilla/src/nasbench301/nasbench301/surrogate_models/utils.py", line 20, in <module>
    from nasbench301.surrogate_models.gnn.gnn import GNNSurrogateModel
  File "/home/ubuntu/naszilla/src/nasbench301/nasbench301/surrogate_models/gnn/gnn.py", line 16, in <module>
    from nasbench301.surrogate_models.gnn.models.deeper_gnn import DeeperGCN
  File "/home/ubuntu/naszilla/src/nasbench301/nasbench301/surrogate_models/gnn/models/deeper_gnn.py", line 8, in <module>
    from nasbench301.surrogate_models.gnn.models.gcn_lib.sparse.torch_nn import norm_layer
  File "/home/ubuntu/naszilla/src/nasbench301/nasbench301/surrogate_models/gnn/models/gcn_lib/sparse/__init__.py", line 1, in <module>
    from .torch_nn import *
  File "/home/ubuntu/naszilla/src/nasbench301/nasbench301/surrogate_models/gnn/models/gcn_lib/sparse/torch_nn.py", line 3, in <module>
    from surrogate_models.gnn.models.gcn_utils.data_util import get_atom_feature_dims, get_bond_feature_dims
ModuleNotFoundError: No module named 'surrogate_models'
    opened by lcmeng 2
  • Path encoding in DARTS search space

    Path encoding in DARTS search space

    Hey @crwhite14:

    The code for generating the path encoding of cell architecture in DARTS seems different from the NASBench-101. Despite the input-to-output paths, the paths that not directly connect to the output also considered to update the path encoding vector. The paths variable in the code stored all the paths; even it does not connect to the output.

    https://github.com/naszilla/naszilla/blob/393e55ea97ee52d2cc79e4b290a05ad301d9401f/naszilla/nas_bench_301/cell_301.py#L220

    I am confused by the code. Is this a bug, or the meaning of the input-to-output path in DARTS search space is different from the NASBench-101?

    opened by auroua 2
  • The performance gap between encode path or not encode path

    The performance gap between encode path or not encode path

    I have run the bananas ablation test many times with the config of encode path=True and encode path=False. I can not get the large performance gap between this two case as in the paper fig5.2. I use the default configuration. The following is the evaluated result. bananas_f means bananas algorithm without encode path. Screenshot_2019-12-17_15-31-35 How could you get such a big performance gap?

    opened by auroua 2
  • Cell301 get_paths function

    Cell301 get_paths function

    Hi,

    I have a question for get_paths function in cell_301.py. In the second loop (cell_301.py -> get_paths function -> Line 180), you are iterating each tuple item in the self.arch. But in this loop, the last element of the list (last edge, ops pair) is not visited. For example, suppose I have an example like the one below.

    self.arch = {[(1, 2), (0, 1), (0, 2), (2, 0), (2, 2), (3, 1), (4, 3), (0, 2)], [(0, 3), (1, 4), (0, 6), (2, 1), (1, 4), (3, 6), (0, 5), (4, 5)]}

    In the inner loop between lines 181 and 193, the last elements of the list are not included because the loop is defined like this:

    for j in range(len(OPS)):

    I think it needs to be changed to:

    for j in range(len(OPS) + 1):

    Thanks,

    Ekran Alıntısı

    opened by zekikus 1
  • Dependency installation error on Ubuntu

    Dependency installation error on Ubuntu

    Dear Naszilla team,

    I am getting this error when I try to install. It seems that pip can no longer find tensorflow 1.14.0 in the standard pathways. This is an Ubuntu 18.04 64 bit system witha P100 gpu.

    (base) dedey@ubuntubox:~/naszilla$ cat requirements.txt | xargs -n 1 -L 1 pip install ERROR: Could not find a version that satisfies the requirement tensorflow-gpu==1.14.0 ERROR: No matching distribution found for tensorflow-gpu==1.14.0 ERROR: Could not find a version that satisfies the requirement tensorflow==1.14.0 ERROR: No matching distribution found for tensorflow==1.14.0

    opened by debadeepta 1
  • Bump tensorflow from 1.14.0 to 2.3.1 in /naszilla

    Bump tensorflow from 1.14.0 to 2.3.1 in /naszilla

    Bumps tensorflow from 1.14.0 to 2.3.1.

    Release notes

    Sourced from tensorflow's releases.

    TensorFlow 2.3.1

    Release 2.3.1

    Bug Fixes and Other Changes

    TensorFlow 2.3.0

    Release 2.3.0

    Major Features and Improvements

    • tf.data adds two new mechanisms to solve input pipeline bottlenecks and save resources:

    In addition checkout the detailed guide for analyzing input pipeline performance with TF Profiler.

    • tf.distribute.TPUStrategy is now a stable API and no longer considered experimental for TensorFlow. (earlier tf.distribute.experimental.TPUStrategy).

    • TF Profiler introduces two new tools: a memory profiler to visualize your model’s memory usage over time and a python tracer which allows you to trace python function calls in your model. Usability improvements include better diagnostic messages and profile options to customize the host and device trace verbosity level.

    • Introduces experimental support for Keras Preprocessing Layers API (tf.keras.layers.experimental.preprocessing.*) to handle data preprocessing operations, with support for composite tensor inputs. Please see below for additional details on these layers.

    • TFLite now properly supports dynamic shapes during conversion and inference. We’ve also added opt-in support on Android and iOS for XNNPACK, a highly optimized set of CPU kernels, as well as opt-in support for executing quantized models on the GPU.

    • Libtensorflow packages are available in GCS starting this release. We have also started to release a nightly version of these packages.

    • The experimental Python API tf.debugging.experimental.enable_dump_debug_info() now allows you to instrument a TensorFlow program and dump debugging information to a directory on the file system. The directory can be read and visualized by a new interactive dashboard in TensorBoard 2.3 called Debugger V2, which reveals the details of the TensorFlow program including graph structures, history of op executions at the Python (eager) and intra-graph levels, the runtime dtype, shape, and numerical composistion of tensors, as well as their code locations.

    Breaking Changes

    • Increases the minimum bazel version required to build TF to 3.1.0.
    • tf.data
      • Makes the following (breaking) changes to the tf.data.
      • C++ API: - IteratorBase::RestoreInternal, IteratorBase::SaveInternal, and DatasetBase::CheckExternalState become pure-virtual and subclasses are now expected to provide an implementation.
      • The deprecated DatasetBase::IsStateful method is removed in favor of DatasetBase::CheckExternalState.
      • Deprecated overrides of DatasetBase::MakeIterator and MakeIteratorFromInputElement are removed.

    ... (truncated)

    Changelog

    Sourced from tensorflow's changelog.

    Release 2.3.1

    Bug Fixes and Other Changes

    Release 2.2.1

    ... (truncated)

    Commits
    • fcc4b96 Merge pull request #43446 from tensorflow-jenkins/version-numbers-2.3.1-16251
    • 4cf2230 Update version numbers to 2.3.1
    • eee8224 Merge pull request #43441 from tensorflow-jenkins/relnotes-2.3.1-24672
    • 0d41b1d Update RELEASE.md
    • d99bd63 Insert release notes place-fill
    • d71d3ce Merge pull request #43414 from tensorflow/mihaimaruseac-patch-1-1
    • 9c91596 Fix missing import
    • f9f12f6 Merge pull request #43391 from tensorflow/mihaimaruseac-patch-4
    • 3ed271b Solve leftover from merge conflict
    • 9cf3773 Merge pull request #43358 from tensorflow/mm-patch-r2.3
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump tensorflow-gpu from 1.14.0 to 2.3.1 in /naszilla

    Bump tensorflow-gpu from 1.14.0 to 2.3.1 in /naszilla

    Bumps tensorflow-gpu from 1.14.0 to 2.3.1.

    Release notes

    Sourced from tensorflow-gpu's releases.

    TensorFlow 2.3.1

    Release 2.3.1

    Bug Fixes and Other Changes

    TensorFlow 2.3.0

    Release 2.3.0

    Major Features and Improvements

    • tf.data adds two new mechanisms to solve input pipeline bottlenecks and save resources:

    In addition checkout the detailed guide for analyzing input pipeline performance with TF Profiler.

    • tf.distribute.TPUStrategy is now a stable API and no longer considered experimental for TensorFlow. (earlier tf.distribute.experimental.TPUStrategy).

    • TF Profiler introduces two new tools: a memory profiler to visualize your model’s memory usage over time and a python tracer which allows you to trace python function calls in your model. Usability improvements include better diagnostic messages and profile options to customize the host and device trace verbosity level.

    • Introduces experimental support for Keras Preprocessing Layers API (tf.keras.layers.experimental.preprocessing.*) to handle data preprocessing operations, with support for composite tensor inputs. Please see below for additional details on these layers.

    • TFLite now properly supports dynamic shapes during conversion and inference. We’ve also added opt-in support on Android and iOS for XNNPACK, a highly optimized set of CPU kernels, as well as opt-in support for executing quantized models on the GPU.

    • Libtensorflow packages are available in GCS starting this release. We have also started to release a nightly version of these packages.

    • The experimental Python API tf.debugging.experimental.enable_dump_debug_info() now allows you to instrument a TensorFlow program and dump debugging information to a directory on the file system. The directory can be read and visualized by a new interactive dashboard in TensorBoard 2.3 called Debugger V2, which reveals the details of the TensorFlow program including graph structures, history of op executions at the Python (eager) and intra-graph levels, the runtime dtype, shape, and numerical composistion of tensors, as well as their code locations.

    Breaking Changes

    • Increases the minimum bazel version required to build TF to 3.1.0.
    • tf.data
      • Makes the following (breaking) changes to the tf.data.
      • C++ API: - IteratorBase::RestoreInternal, IteratorBase::SaveInternal, and DatasetBase::CheckExternalState become pure-virtual and subclasses are now expected to provide an implementation.
      • The deprecated DatasetBase::IsStateful method is removed in favor of DatasetBase::CheckExternalState.
      • Deprecated overrides of DatasetBase::MakeIterator and MakeIteratorFromInputElement are removed.

    ... (truncated)

    Changelog

    Sourced from tensorflow-gpu's changelog.

    Release 2.3.1

    Bug Fixes and Other Changes

    Release 2.2.1

    ... (truncated)

    Commits
    • fcc4b96 Merge pull request #43446 from tensorflow-jenkins/version-numbers-2.3.1-16251
    • 4cf2230 Update version numbers to 2.3.1
    • eee8224 Merge pull request #43441 from tensorflow-jenkins/relnotes-2.3.1-24672
    • 0d41b1d Update RELEASE.md
    • d99bd63 Insert release notes place-fill
    • d71d3ce Merge pull request #43414 from tensorflow/mihaimaruseac-patch-1-1
    • 9c91596 Fix missing import
    • f9f12f6 Merge pull request #43391 from tensorflow/mihaimaruseac-patch-4
    • 3ed271b Solve leftover from merge conflict
    • 9cf3773 Merge pull request #43358 from tensorflow/mm-patch-r2.3
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Does Python version matter？

    Does Python version matter？

    I create the project environment on Anaconda with python=3.7, and while following the requirements.txt to install packages there occurred some problem while installing torch-scatter, torch-sparse, torch-cluster, torch-spline-conv. Finally, I use "wget" to directly download torch-***. After that, I downloaded the three NasBenchmark datasets following the three wget commands, and then I follow the "Test Installation" with "python naszilla/run_experiments.py --search_space nasbench_101 --algo_params all_algos --queries 30 --trials 1". While running the above run_experiments.py the programm ended with the following statement:

    " File "/home/umin/.conda/envs/SXH_AUTOML/lib/python3.7/site-packages/torch_geometric/loader/dynamic_batch_sampler.py", line 9, in class DynamicBatchSampler(torch.utils.data.sampler.Sampler[List[int]]): TypeError: 'type' object is not subscriptable"

    So, is this just because of my wrongly configuring the project environment?

    opened by llstela 0
  • RuntimeError with GCN predictor in installation test

    RuntimeError with GCN predictor in installation test

    Command: python naszilla/run_experiments.py --search_space nasbench_101 --algo_params all_algos --queries 30 --trials 1 Log:

    ...
Finished GP-BayesOpt query 0
Finished GP-BayesOpt query 10

* Running NAS algorithm: {'algo_name': 'gcn_predictor', 'total_queries': 30}
Traceback (most recent call last):
  File "naszilla/run_experiments.py", line 121, in <module>
    main(args)
  File "naszilla/run_experiments.py", line 104, in main
    run_experiments(args, save_path)
  File "naszilla/run_experiments.py", line 52, in run_experiments
    result, val_result, run_datum = run_nas_algorithm(algorithm_params[j], search_space, mp)
  File "/home/ubuntu/naszilla/naszilla/nas_algorithms.py", line 47, in run_nas_algorithm
    data = gcn_predictor(search_space, **ps)
  File "/home/ubuntu/naszilla/naszilla/nas_algorithms.py", line 420, in gcn_predictor
    fit(net, xtrain, seed=seed)
  File "/home/ubuntu/naszilla/naszilla/gcn/train_gcn.py", line 53, in fit
    prediction = net(batch)
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/naszilla/naszilla/gcn/model.py", line 66, in forward
    out = layer(out, adj_with_diag)
  File "/home/ubuntu/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/naszilla/naszilla/gcn/model.py", line 37, in forward
    output1 = F.relu(torch.matmul(norm_adj, torch.matmul(inputs, self.weight1)))
RuntimeError: Expected object of device type cuda but got device type cpu for argument #1 'self' in call to _th_mm

    This happened on both single-GPU and multi-GPU machines.

    opened by lcmeng 2
  • Val Accuracy prediction with D-VAE

    Val Accuracy prediction with D-VAE

    Hey @crwhite14

    how do you predict val accuracies with D-VAE? Do you use a similar setting as in the experiment in section 4.2 of the D-VAE paper (i.e. using a sparse GP)?

    opened by infomon 1
Owner
null
GitHub
"NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search".

NAS-Bench-301 This repository containts code for the paper: "NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search". The

AutoML-Freiburg-Hannover 57 Nov 30, 2022
CM-NAS: Cross-Modality Neural Architecture Search for Visible-Infrared Person Re-Identification (ICCV2021)

CM-NAS Official Pytorch code of paper CM-NAS: Cross-Modality Neural Architecture Search for Visible-Infrared Person Re-Identification in ICCV2021. Vis

JDAI-CV 40 Nov 25, 2022
NAS Benchmark in "Prioritized Architecture Sampling with Monto-Carlo Tree Search", CVPR2021

NAS-Bench-Macro This repository includes the benchmark and code for NAS-Bench-Macro in paper "Prioritized Architecture Sampling with Monto-Carlo Tree

null 35 Jan 3, 2023
code for paper "Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?"

Does Unsupervised Architecture Representation Learning Help Neural Architecture Search? Code for paper: Does Unsupervised Architecture Representation

null 39 Dec 17, 2022
Densely Connected Search Space for More Flexible Neural Architecture Search (CVPR2020)

DenseNAS The code of the CVPR2020 paper Densely Connected Search Space for More Flexible Neural Architecture Search. Neural architecture search (NAS)

Jamin Fong 291 Nov 18, 2022
[CVPR 2021] 'Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator'

[CVPR2021] Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator Overview This is the entire codebase for the paper

null 35 Dec 1, 2022
[ICLR2021oral] Rethinking Architecture Selection in Differentiable NAS

DARTS-PT Code accompanying the paper ICLR'2021: Rethinking Architecture Selection in Differentiable NAS Ruochen Wang, Minhao Cheng, Xiangning Chen, Xi

Ruochen Wang 86 Dec 27, 2022
DeepHyper: Scalable Asynchronous Neural Architecture and Hyperparameter Search for Deep Neural Networks

What is DeepHyper? DeepHyper is a software package that uses learning, optimization, and parallel computing to automate the design and development of

DeepHyper Team 214 Jan 8, 2023
Model search is a framework that implements AutoML algorithms for model architecture search at scale

Model search (MS) is a framework that implements AutoML algorithms for model architecture search at scale. It aims to help researchers speed up their exploration process for finding the right model architecture for their classification problems (i.e., DNNs with different types of layers).

Google 3.2k Dec 31, 2022
Deep Image Search is an AI-based image search engine that includes deep transfor learning features Extraction and tree-based vectorized search.

Deep Image Search - AI-Based Image Search Engine Deep Image Search is an AI-based image search engine that includes deep transfer learning features Ex

null 139 Jan 1, 2023
[ICLR 2021] "Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective" by Wuyang Chen, Xinyu Gong, Zhangyang Wang

Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective [PDF] Wuyang Chen, Xinyu Gong, Zhangyang Wang In ICLR 2

VITA 156 Nov 28, 2022
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search

BossNAS This repository contains PyTorch evaluation code, retraining code and pretrained models of our paper: BossNAS: Exploring Hybrid CNN-transforme

Changlin Li 127 Dec 26, 2022
Deep Multimodal Neural Architecture Search

MMNas: Deep Multimodal Neural Architecture Search This repository corresponds to the PyTorch implementation of the MMnas for visual question answering

Vision and Language Group@ MIL 23 Dec 21, 2022
Official implementation of Rethinking Graph Neural Architecture Search from Message-passing (CVPR2021)

Rethinking Graph Neural Architecture Search from Message-passing Intro The GNAS can automatically learn better architecture with the optimal depth of

Shaofei Cai 48 Sep 30, 2022
Block-wisely Supervised Neural Architecture Search with Knowledge Distillation (CVPR 2020)

DNA This repository provides the code of our paper: Blockwisely Supervised Neural Architecture Search with Knowledge Distillation. Illustration of DNA

Changlin Li 215 Dec 19, 2022
[CVPR21] LightTrack: Finding Lightweight Neural Network for Object Tracking via One-Shot Architecture Search

LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search The official implementation of the paper LightTra

Multimedia Research 290 Dec 24, 2022
Code release to accompany paper "Geometry-Aware Gradient Algorithms for Neural Architecture Search."

Geometry-Aware Gradient Algorithms for Neural Architecture Search This repository contains the code required to run the experiments for the DARTS sear

null 18 May 27, 2022
Official PyTorch implementation of "Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets" (ICLR 2021)

Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets This is the official PyTorch implementation for the paper Rapid Neural A

null 48 Dec 26, 2022
code for "AttentiveNAS Improving Neural Architecture Search via Attentive Sampling"

code for "AttentiveNAS Improving Neural Architecture Search via Attentive Sampling"

Facebook Research 94 Oct 26, 2022