EfficientNetV2 implementation using PyTorch

Very impressive project ! As your readme motioned. How the metric get by the code.

Number of parameters: 23941296
Time per operator type:
        778.049 ms.    70.6258%. Conv
        255.227 ms.    23.1677%. Sigmoid
          56.91 ms.    5.16589%. Mul
         6.1573 ms.   0.558916%. Add
        4.69289 ms.   0.425987%. ReduceMean
       0.613303 ms.  0.0556713%. FC
        1101.65 ms in Total
FLOP per operator type:
         17.277 GFLOP.    99.7074%. Conv
      0.0419251 GFLOP.   0.241954%. Mul
     0.00519322 GFLOP.  0.0299706%. Add
       0.003585 GFLOP.  0.0206894%. FC
        17.3277 GFLOP in Total
Feature Memory Read per operator type:
        295.875 MB.    50.5134%. Mul
        241.136 MB.     41.168%. Conv
        41.5457 MB.     7.0929%. Add
        7.17917 MB.    1.22567%. FC
        585.737 MB in Total
Feature Memory Written per operator type:
          167.7 MB.    49.2361%. Mul
        152.127 MB.    44.6639%. Conv
        20.7729 MB.    6.09882%. Add
          0.004 MB. 0.00117438%. FC
        340.605 MB in Total
Parameter Memory per operator type:
        87.8034 MB.    92.4486%. Conv
          7.172 MB.    7.55143%. FC
              0 MB.          0%. Add
              0 MB.          0%. Mul
        94.9754 MB in Total

HARDWARE

• Ubuntu 18.04
• Torch 1.7.0a0+7036e91
• Torch vision 0.8.0a0

Hi, @jahongir7174 Thank you for your awesome work.

I tried to run a test on your pre-trained model. However, It seems that the pre-trained model gives me an error. Could you please help me to solve this problem?

Number of parameters: 21458488
     acc@1     acc@5:   0%|                                                                                                                                                          | 0/48 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "main.py", line 227, in <module>
    main()
  File "main.py", line 223, in main
    test()
  File "main.py", line 171, in test
    loss, acc1, acc5 = batch(images, target, model, criterion)
  File "main.py", line 43, in batch
    output = model(images)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/ws/EfficientNetV2/nets/nn.py", line 164, in forward
    x = self.p1(x)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/container.py", line 117, in forward
    input = module(input)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/ws/EfficientNetV2/nets/nn.py", line 69, in forward
    return x + self.res(x) if self.add else self.res(x)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/container.py", line 117, in forward
    input = module(input)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/timm/models/layers/drop.py", line 164, in forward
    return drop_path(x, self.drop_prob, self.training, self.scale_by_keep)
  File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 779, in __getattr__
    type(self).__name__, name))
torch.nn.modules.module.ModuleAttributeError: 'DropPath' object has no attribute 'scale_by_keep'

I also create a script to test your model on a single image, got the same problem too.

import torch
from torchvision import transforms
from PIL import Image


model = torch.load('weights/best.pt', map_location='cuda')['model'].float()
with torch.no_grad():
    model.eval()  

image = Image.open("/ws/dataset/hamxe_dataset/data_fake/data_Fake/data/mau/bike/EtzcWfSbQS.jpg")
normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])

transform_norm = transforms.Compose([transforms.Resize(384),
                                          transforms.CenterCrop(384),
                                          transforms.ToTensor(), normalize])
img_normalized = transform_norm(image).float()
img_normalized = img_normalized.unsqueeze_(0)
img_normalized = img_normalized.cuda()

with torch.cuda.amp.autocast():
    output = model(img_normalized)

print(output)

Hi, thanks for your wonderful code, may I ask if I can have your permission to fork you code and add arguments for your training, that would improve on how easy it is to use the script?

A few additional questions?

• Do I have to specify number of classes I need to train
• Have you compared your result with timm library yet?

In your python file nets/nn.py. I think your class SE for realize SE(squeeze and Excitation).Why your init() haven't avgpool for squeeze,And self.se use conv2d. did not nn.linear(). Looking forward to your reply.