scalingscattering

Overview

Scaling The Scattering Transform : Deep Hybrid Networks

This repository contains the experiments found in the paper: https://arxiv.org/abs/1703.08961

Requirements

In order to run our experiments you will need at minimum the following python packages: pytorch,opencv,pyscatwave package. The simplest way to install pytorch and opencv is through anaconda. We recommend python 2.7 + anaconda. The pyscatwave package can be found here https://github.com/edouardoyallon/pyscatwave

Imagenet

We provide a pre-trained model similar to the one described in the paper.

To run the trained model of scattering+resnet on imagenet ILSVRC validation set:

  1. Make sure you have downloaded at least the validation set of ILSVRC2012 and have it organized by class categories Note: due to problems with pytorch dataset constructors make sure your imagenet directory has no hidden files, or extra directories besides the 1000 ILSVRC categories.. otherwise all the images will be mislabeled
  2. Download the model file from http://www.di.ens.fr/~oyallon/scatter_resnet_10_model.pt7
  3. Add this to the imagenet/ directory
  4. Run the script main_test.py to evaluate on the ILSVRC validation set specifying --imagenetpath to point to your imagenet directory

Training scripts for imagenet and SLE feature extractor will be added soon

STL-10

Simply run python main_STL.py script in the STL directory

CIFAR-10

To run the small sample experiments Example:

python main_small_sample_class_normalized.py --model resnet12_8_scat --save "test"  --seed 1 --sampleSize 500 --mul 20
Comments
  • no results

    no results

    hi ! thanks for your great job ! I debugede your code , but after this result: "Total number of parameters: 12777726.000000" the code stopped running,I waited for a few hours ,but still haven't any change, do you know the reason?

    opened by airsimonhan 11
  • New version of pyscatwave

    New version of pyscatwave

    First, congratulation for this very interesting work. I would be interested in tested hybrid networks for 1D specific signals. When do you plan the new version you mention in pyscatwave description, including "1D-2D-3D fast, optimized, differentiable Scattering Transform" ?

    opened by ghost 1
  • About the inverse operation for cifar-dataset

    About the inverse operation for cifar-dataset

    Hello,

    Thanks a lot for your generous sharing.

    I want to use your proposed structure in some adversarial training framework.

    My problem is that:

    When batch size is 64, the previous data has the size of 64332*32

    For J=1, after scattering, the matrix size is 64391616. For J=2, after scattering, the matrix size is 6431788. For J=3, after scattering, the matrix size is 6432544. For J=4, after scattering, the matrix size is 6433322.

    Looks like the tensor-shape for third dimesion is 1+J*8, the fourth and fifth dimension is 32/2^j.

    I am not sure, is there a inverse operation API that I can restore the matrix after scattering to the original matrix, e.g., Tensor with size 64332*32.

    It would be honor if you could answer the questions. Thanks & regards! Momo

    opened by momo1986 0
  • SLE implementation

    SLE implementation

    Training scripts for imagenet and SLE feature extractor will be added soon

    I took a quick look at the models defined in each example and it seems that SLE is not implemented in anyone yet. I was wondering if it actually is not implemented or I can't find it.

    opened by nshervt 0
  • Deprecated modules

    Deprecated modules

    I tried using kymatio with this code. If you were interested in updating it, I switched the scattering method to kymatio but there are more things to be fixed, for instance, in STL, it seems that torch.legacy that you are using in utils has been deprecated and needs to be changed. In imagenet test, line torch.randn(8).cuda() again uses cuda, and when I comment .cuda() out, I fall into a rabbit hole of errors. the latest being

    Traceback (most recent call last):
      File "/Users/Nshervt/Desktop/scalingscattering-master/imagenet/main_test.py", line 186, in <module>
        main()
      File "/Users/Nshervt/Desktop/scalingscattering-master/imagenet/main_test.py", line 65, in main
        model, params, stats = models.__dict__[opt.model](N=opt.N,J=opt.scat)
      File "/Users/Nshervt/Desktop/scalingscattering-master/imagenet/models/scatter_resnet.py", line 151, in scatresnet6_2
        model = ScatResNet(J,N)
      File "/Users/Nshervt/Desktop/scalingscattering-master/imagenet/models/scatter_resnet.py", line 61, in __init__
        self.bn0 = nn.BatchNorm2d(3*self.nfscat,eps=1e-5, momentum=0.9, affine=False)
      File "/Users/Nshervt/anaconda3/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 34, in __init__
        self.register_buffer('running_mean', torch.zeros(num_features))
    TypeError: zeros(): argument 'size' (position 1) must be tuple of ints, not float
    
    opened by nshervt 4
Owner
Edouard Oyallon
Applied maths guy
Edouard Oyallon