EfficientNetV2-with-TPU - Cifar-10 case study

Sultan syach

Last update: Dec 28, 2021

Related tags

Deep Learning EfficientNetV2-with-TPU

Overview

EfficientNetV2-with-TPU

EfficientNet

EfficientNetV2 adalah jenis jaringan saraf convolutional yang memiliki kecepatan pelatihan lebih cepat dan efisiensi parameter yang lebih baik dari model sebelumnya . Untuk mengembangkan model ini, penulis menggunakan kombinasi pencarian dan penskalaan arsitektur saraf yang sadar pelatihan , untuk bersama-sama mengoptimalkan kecepatan pelatihan. Model dicari dari ruang pencarian yang diperkaya dengan operasi baru seperti Fused-MBConv .

Secara arsitektur perbedaan utama adalah:

EfficientNetV2 secara ekstensif menggunakan MBConv dan fusi-MBConv yang baru ditambahkan di lapisan awal.
EfficientNetV2 lebih memilih rasio ekspansi yang lebih kecil untuk MBConv karena rasio ekspansi yang lebih kecil cenderung memiliki lebih sedikit overhead akses memori.
EfficientNetV2 lebih menyukai ukuran kernel 3x3 yang lebih kecil, tetapi menambahkan lebih banyak lapisan untuk mengkompensasi bidang reseptif yang berkurang yang dihasilkan dari ukuran kernel yang lebih kecil.
EfficientNetV2 sepenuhnya menghapus tahap stride-1 terakhir di EfficientNet asli, mungkin karena ukuran parameternya yang besar dan overhead akses memori

Note

Model	Size	acc-val	top-5	acc-test	weight
EfficientNetV2B0	224	90.68	99.76	89.86	imagenet
EfficientNetV2B1	240	90.76	99.78	90.07	imagenet
EfficientNetV2B2	260	87.08	99.48	86.85	imagenet
EfficientNetV2B3	300	90.38	99.80	89.29	imagenet
EfficientNetV2T	320	92.80	99.86	92.53	imagenet
EfficientNetV2S	384	89.94	99.74	89.27	imagenet
EfficientNetV2M	480	91.86	99.70	90.53	imagenet
EfficientNetV2L	480	93.10	99.80	92.38	imagenet
EfficientNetV2XL	512	93.24	99.72	93.41	imagenet21K-ft1k

Train 90%(45000rb)
Validation 10%(5000rb)
Test(10000rb)
Epochs = 25
WeightDecay = 1e-5
Batchsize = 16 * 8(strategy.num_replicas_in_sync)
optimizers adabelief dengan LearningRateSchduler(Triangular2CyclicalLearningRate) dan Rectified = True(mencegah overshoot)
cifar-10 tidak di sarankan untuk di ubah ukuran nya, saya mengubah ukuran nya hanya untuk milihat apakah bagus/tidak efficientnetv2 saat mempelajari cifar-10

Referensi

The full training script for Enformer (Tensorflow Sonnet) on TPU clusters

Enformer TPU training script (wip) The full training script for Enformer (Tensorflow Sonnet) on TPU clusters, in an effort to migrate the model to pyt

10 Oct 19, 2022

Implementation of Squeezenet in pytorch, pretrained models on Cifar 10 data to come

Pytorch Squeeznet Pytorch implementation of Squeezenet model as described in https://arxiv.org/abs/1602.07360 on cifar-10 Data. The definition of Sque

86 Oct 28, 2022

TensorFlow2 Classification Model Zoo playing with TensorFlow2 on the CIFAR-10 dataset.

Training CIFAR-10 with TensorFlow2(TF2) TensorFlow2 Classification Model Zoo. I'm playing with TensorFlow2 on the CIFAR-10 dataset. Architectures LeNe

16 Sep 27, 2022

Training Cifar-10 Classifier Using VGG16

opevcvdl-hw3 This project uses pytorch and Qt to achieve the requirements. Version Python 3.6 opencv-contrib-python 3.4.2.17 Matplotlib 3.1.1 pyqt5 5.

3 Aug 17, 2022

Training a deep learning model on the noisy CIFAR dataset

Training-a-deep-learning-model-on-the-noisy-CIFAR-dataset This repository contai

1 Jun 14, 2022

Everything you want about DP-Based Federated Learning, including Papers and Code. (Mechanism: Laplace or Gaussian, Dataset: femnist, shakespeare, mnist, cifar-10 and fashion-mnist. )

Differential Privacy (DP) Based Federated Learning (FL) Everything about DP-based FL you need is here. （所有你需要的DP-based FL的信息都在这里） Code Tip: the code o

83 Dec 24, 2022

This repository is related to an Arabic tutorial, within the tutorial we discuss the common data structure and algorithms and their worst and best case for each, then implement the code using Python.

Data Structure and Algorithms with Python This repository is related to the Arabic tutorial here, within the tutorial we discuss the common data struc

33 Dec 2, 2022

Code for the paper: Learning Adversarially Robust Representations via Worst-Case Mutual Information Maximization (https://arxiv.org/abs/2002.11798)

Representation Robustness Evaluations Our implementation is based on code from MadryLab's robustness package and Devon Hjelm's Deep InfoMax. For all t

19 Dec 7, 2022

"NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search".

NAS-Bench-301 This repository containts code for the paper: "NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search". The

57 Nov 30, 2022

EfficientNetV2-with-TPU - Cifar-10 case study

Related tags

Overview

EfficientNetV2-with-TPU

Note

Referensi

You might also like...

The full training script for Enformer (Tensorflow Sonnet) on TPU clusters

Implementation of Squeezenet in pytorch, pretrained models on Cifar 10 data to come

TensorFlow2 Classification Model Zoo playing with TensorFlow2 on the CIFAR-10 dataset.

Training Cifar-10 Classifier Using VGG16

Training a deep learning model on the noisy CIFAR dataset

Everything you want about DP-Based Federated Learning, including Papers and Code. (Mechanism: Laplace or Gaussian, Dataset: femnist, shakespeare, mnist, cifar-10 and fashion-mnist. )

This repository is related to an Arabic tutorial, within the tutorial we discuss the common data structure and algorithms and their worst and best case for each, then implement the code using Python.

Code for the paper: Learning Adversarially Robust Representations via Worst-Case Mutual Information Maximization (https://arxiv.org/abs/2002.11798)

"NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search".

Owner

Sultan syach

CIFAR-10_train-test - training and testing codes for dataset CIFAR-10

Network Pruning That Matters: A Case Study on Retraining Variants (ICLR 2021)

PyTorch implementation of EfficientNetV2

EfficientNetv2 TensorRT int8

EfficientNetV2 implementation using PyTorch

A PyTorch implementation of EfficientNet and EfficientNetV2 (coming soon!)

Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

Refactoring dalle-pytorch and taming-transformers for TPU VM

A minimal TPU compatible Jax implementation of NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis