Submanifold sparse convolutional networks

Overview

Submanifold Sparse Convolutional Networks

This is the PyTorch library for training Submanifold Sparse Convolutional Networks.

Spatial sparsity

This library brings Spatially-sparse convolutional networks to PyTorch. Moreover, it introduces Submanifold Sparse Convolutions, that can be used to build computationally efficient sparse VGG/ResNet/DenseNet-style networks.

With regular 3x3 convolutions, the set of active (non-zero) sites grows rapidly:
submanifold
With Submanifold Sparse Convolutions, the set of active sites is unchanged. Active sites look at their active neighbors (green); non-active sites (red) have no computational overhead:
submanifold
Stacking Submanifold Sparse Convolutions to build VGG and ResNet type ConvNets, information can flow along lines or surfaces of active points.

Disconnected components don't communicate at first, although they will merge due to the effect of strided operations, either pooling or convolutions. Additionally, adding ConvolutionWithStride2-SubmanifoldConvolution-DeconvolutionWithStride2 paths to the network allows disjoint active sites to communicate; see the 'VGG+' networks in the paper.
Strided Convolution, convolution, deconvolution
Strided Convolution, convolution, deconvolution
From left: (i) an active point is highlighted; a convolution with stride 2 sees the green active sites (ii) and produces output (iii), 'children' of hightlighted active point from (i) are highlighted; a submanifold sparse convolution sees the green active sites (iv) and produces output (v); a deconvolution operation sees the green active sites (vi) and produces output (vii).

Dimensionality and 'submanifolds'

SparseConvNet supports input with different numbers of spatial/temporal dimensions. Higher dimensional input is more likely to be sparse because of the 'curse of dimensionality'.

Dimension Name in 'torch.nn' Use cases
1 Conv1d Text, audio
2 Conv2d Lines in 2D space, e.g. handwriting
3 Conv3d Lines and surfaces in 3D space or (2+1)D space-time
4 - Lines, etc, in (3+1)D space-time

We use the term 'submanifold' to refer to input data that is sparse because it has a lower effective dimension than the space in which it lives, for example a one-dimensional curve in 2+ dimensional space, or a two-dimensional surface in 3+ dimensional space.

In theory, the library supports up to 10 dimensions. In practice, ConvNets with size-3 SVC convolutions in dimension 5+ may be impractical as the number of parameters per convolution is growing exponentially. Possible solutions include factorizing the convolutions (e.g. 3x1x1x..., 1x3x1x..., etc), or switching to a hyper-tetrahedral lattice (see Sparse 3D convolutional neural networks).

Hello World

SparseConvNets can be built either by defining a function that inherits from torch.nn.Module or by stacking modules in a sparseconvnet.Sequential:

import torch
import sparseconvnet as scn

# Use the GPU if there is one, otherwise CPU
device = 'cuda:0' if torch.cuda.is_available() else 'cpu'

model = scn.Sequential().add(
    scn.SparseVggNet(2, 1,
                     [['C', 8], ['C', 8], ['MP', 3, 2],
                      ['C', 16], ['C', 16], ['MP', 3, 2],
                      ['C', 24], ['C', 24], ['MP', 3, 2]])
).add(
    scn.SubmanifoldConvolution(2, 24, 32, 3, False)
).add(
    scn.BatchNormReLU(32)
).add(
    scn.SparseToDense(2, 32)
).to(device)

# output will be 10x10
inputSpatialSize = model.input_spatial_size(torch.LongTensor([10, 10]))
input_layer = scn.InputLayer(2, inputSpatialSize)

msgs = [[" X   X  XXX  X    X    XX     X       X   XX   XXX   X    XXX   ",
         " X   X  X    X    X   X  X    X       X  X  X  X  X  X    X  X  ",
         " XXXXX  XX   X    X   X  X    X   X   X  X  X  XXX   X    X   X ",
         " X   X  X    X    X   X  X     X X X X   X  X  X  X  X    X  X  ",
         " X   X  XXX  XXX  XXX  XX       X   X     XX   X  X  XXX  XXX   "],

        [" XXX              XXXXX      x   x     x  xxxxx  xxx ",
         " X  X  X   XXX       X       x   x x   x  x     x  x ",
         " XXX                X        x   xxxx  x  xxxx   xxx ",
         " X     X   XXX       X       x     x   x      x    x ",
         " X     X          XXXX   x   x     x   x  xxxx     x ",]]


# Create Nx3 and Nx1 vectors to encode the messages above:
locations = []
features = []
for batchIdx, msg in enumerate(msgs):
    for y, line in enumerate(msg):
        for x, c in enumerate(line):
            if c == 'X':
                locations.append([y, x, batchIdx])
                features.append([1])
locations = torch.LongTensor(locations)
features = torch.FloatTensor(features).to(device)

input = input_layer([locations,features])
print('Input SparseConvNetTensor:', input)
output = model(input)

# Output is 2x32x10x10: our minibatch has 2 samples, the network has 32 output
# feature planes, and 10x10 is the spatial size of the output.
print('Output SparseConvNetTensor:', output)

Examples

Examples in the examples folder include

For example:

cd examples/Assamese_handwriting
python VGGplus.py

Setup

Tested with PyTorch 1.3, CUDA 10.0, and Python 3.3 with Conda.

conda install pytorch torchvision cudatoolkit=10.0 -c pytorch # See https://pytorch.org/get-started/locally/
git clone [email protected]:facebookresearch/SparseConvNet.git
cd SparseConvNet/
bash develop.sh

To run the examples you may also need to install unrar:

apt-get install unrar

License

SparseConvNet is BSD licensed, as found in the LICENSE file. Terms of use. Privacy

Links

  1. ICDAR 2013 Chinese Handwriting Recognition Competition 2013 First place in task 3, with test error of 2.61%. Human performance on the test set was 4.81%. Report
  2. Spatially-sparse convolutional neural networks, 2014 SparseConvNets for Chinese handwriting recognition
  3. Fractional max-pooling, 2014 A SparseConvNet with fractional max-pooling achieves an error rate of 3.47% for CIFAR-10.
  4. Sparse 3D convolutional neural networks, BMVC 2015 SparseConvNets for 3D object recognition and (2+1)D video action recognition.
  5. Kaggle plankton recognition competition, 2015 Third place. The competition solution is being adapted for research purposes in EcoTaxa.
  6. Kaggle Diabetic Retinopathy Detection, 2015 First place in the Kaggle Diabetic Retinopathy Detection competition.
  7. Submanifold Sparse Convolutional Networks, 2017 Introduces deep 'submanifold' SparseConvNets.
  8. Workshop on Learning to See from 3D Data, 2017 First place in the semantic segmentation competition. Report
  9. 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks, 2017 Semantic segmentation for the ShapeNet Core55 and NYU-DepthV2 datasets, CVPR 2018
  10. Unsupervised learning with sparse space-and-time autoencoders (3+1)D space-time autoencoders
  11. ScanNet 3D semantic label benchmark 2018 0.726 average IOU.
  12. MinkowskiEngine is an alternative implementation of SparseConvNet; 0.736 average IOU for ScanNet.
  13. SpConv: PyTorch Spatially Sparse Convolution Library is an alternative implementation of SparseConvNet.
  14. Live Semantic 3D Perception for Immersive Augmented Reality describes a way to optimize memory access for SparseConvNet.
  15. OccuSeg real-time object detection using SparseConvNets.
  16. TorchSparse implements 3D submanifold convolutions.
  17. TensorFlow 3D implements submanifold convolutions.

Citations

If you find this code useful in your research then please cite:

3D Semantic Segmentation with Submanifold Sparse Convolutional Networks, CVPR 2018
Benjamin Graham,
Martin Engelcke,
Laurens van der Maaten,

@article{3DSemanticSegmentationWithSubmanifoldSparseConvNet,
  title={3D Semantic Segmentation with Submanifold Sparse Convolutional Networks},
  author={Graham, Benjamin and Engelcke, Martin and van der Maaten, Laurens},
  journal={CVPR},
  year={2018}
}

and/or

Submanifold Sparse Convolutional Networks, https://arxiv.org/abs/1706.01307
Benjamin Graham,
Laurens van der Maaten,

@article{SubmanifoldSparseConvNet,
  title={Submanifold Sparse Convolutional Networks},
  author={Graham, Benjamin and van der Maaten, Laurens},
  journal={arXiv preprint arXiv:1706.01307},
  year={2017}
}
Comments
  • How fast is training on the scannet benchmark?

    How fast is training on the scannet benchmark?

    I am training a large network on the ScanNet dataset with the parameters specified in the Readme.md However, I find the training to be really slow. I am training on 8 v100 GPUs with a batch_size of 32 and a single epoch takes about 35 minutes. I would like to know how long training took to achieve the benchmark? and also, what hardware was used in the training. Thanks!

    opened by micaeltchapmi 12
  • how to specify padding?

    how to specify padding?

    i don't see where to specify padding for convolutions? when i create a dummy model with one layer, filter_size=3, filter_stride=1 and compute input_spatial_size for [10,10] output i get 12, 12, which seems to indicate padding = 0.

    opened by etienne87 10
  • There seems a bug for 1x1 convolution

    There seems a bug for 1x1 convolution

    Hello. I'm now using the SparseConvNet for character regconition. And I found when I set the filter size to 1 for the scn.Convolution layer, then the layer had empty output. I did not dive deep into the source code. Is this a bug? How can I fix it?

    opened by LaiSongxuan 8
  • Average- and MaxPooling behave strangely

    Average- and MaxPooling behave strangely

    I'm trying to build a squeeze/excite-style block. Unpooling is undesirable because of all the blank space, but it runs without complaint. However, Deconvolution after Average- or MaxPooling fails. It returns tensor.features of shape [0, N_features] where the 0 was previously the number of input points. AveragePooling outputs features that don't look obviously crazy and spatial_size works correctly for all of the above, but tensor.get_spatial_locations() is empty after pooling. I believe that is the reason the Deconvolution layer fails but I'm not positive. Strangely tensor.metadata.getSpatialLocations(tensor.spatial_size) is empty, but tensor.metadata.getSpatialLocations(tensor.spatial_size before pool operation) seems to give the desired answer. I can't find any obvious crossed wires in the python wrappers so it seems likely the problem is in the metadata or pooling CUDA files. I'll update if I find it.

    opened by HughRunyan 7
  • Assertion error scn.convolution kernel 2 stride 1

    Assertion error scn.convolution kernel 2 stride 1

    Hi, I am trying to perform scn.convolution with a stride of 1 and kernel size of 2 but I get an assertion error. I would like to know if that is possible and how it could be implemented.

    opened by micaeltchapmi 7
  • AttributeError: 'int' object has no attribute 'item'

    AttributeError: 'int' object has no attribute 'item'

    hi, when i run hello_world.py, I got a error: File "/home/XXX/software/anaconda2/envs/pytorch_py3/lib/python3.6/site-packages/sparseconvnet-0.1.1-py3.6.egg/sparseconvnet/submanifoldConvolution.py", line 22, in init self.filter_volume = self.filter_size.prod().item() AttributeError: 'int' object has no attribute 'item' How can I deal with. Thanks

    opened by fengxiuyaun 7
  • Python Module Failing

    Python Module Failing

    Hi,

    First, great work! However I'm trying to get the Python examples running but hitting this error:

    Traceback (most recent call last):
      File "hello-world.py", line 8, in <module>
        import sparseconvnet.legacy as scn
      File "/.../local/lib/python2.7/site-packages/sparseconvnet/legacy/__init__.py", line 7, in <module>
        from ..utils import *
      File /.../local/lib/python2.7/site-packages/sparseconvnet/utils.py", line 8, in <module>
        import sparseconvnet.SCN as scn
      File "/.../local/lib/python2.7/site-packages/sparseconvnet/SCN/__init__.py", line 3, in <module>
        from ._SCN import lib as _lib, ffi as _ffi
    ImportError: No module named _SCN
    

    Could this be a missing config in setup.py? Seems like the ._SCN module isn't built or copied over. Any help is appreciated.

    Cheers

    opened by gnedster 7
  • Jupyter lab integration

    Jupyter lab integration

    the test script 'hello-world.py' works perfectly, but I cannot run the same script on jupyter lab. I think there is an issue with pybind, but I could not resolve it. Basically, it gives these errors:

    `--------------------------------------------------------------------------- AttributeError Traceback (most recent call last) in 4 5 # Use the GPU if there is one and sparseconvnet can use it, otherwise CPU ----> 6 use_cuda = torch.cuda.is_available() and scn.SCN.is_cuda_build() 7 device = 'cuda:0' if use_cuda else 'cpu' 8 if use_cuda:

    AttributeError: module 'sparseconvnet.SCN' has no attribute 'is_cuda_build'`

    `~/Documents/sparse_conv/SparseConvNet/sparseconvnet/metadata.py in Metadata(dim) 15 16 def Metadata(dim): ---> 17 return getattr(sparseconvnet.SCN, 'Metadata_%d'%dim)()

    AttributeError: module 'sparseconvnet.SCN' has no attribute 'Metadata_2'`

    opened by evinpinar 6
  • unable to execute ':/usr/local/cuda-8.0/bin/nvcc': No such file or directory

    unable to execute ':/usr/local/cuda-8.0/bin/nvcc': No such file or directory

    when i try to install SparseConvNet,By “bash build.sh”

    running install running bdist_egg running egg_info creating sparseconvnet.egg-info writing sparseconvnet.egg-info/PKG-INFO writing dependency_links to sparseconvnet.egg-info/dependency_links.txt writing top-level names to sparseconvnet.egg-info/top_level.txt writing manifest file 'sparseconvnet.egg-info/SOURCES.txt' package init file 'sparseconvnet/SCN/init.py' not found (or not a regular file) reading manifest file 'sparseconvnet.egg-info/SOURCES.txt' writing manifest file 'sparseconvnet.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib.linux-x86_64-3.7 creating build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/denseToSparse.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/permutohedralSubmanifoldConvolution.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/randomizedStrideConvolution.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/sparseConvNetTensor.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/submanifoldConvolution.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/maxPooling.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/networkArchitectures.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/classificationTrainValidate.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/activations.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/identity.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/utils.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/dropout.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/ioLayers.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/batchNormalization.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/metadata.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/deconvolution.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/shapeContext.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/sparsify.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/convolution.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/averagePooling.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/inputBatch.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/networkInNetwork.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/tables.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/randomizedStrideMaxPooling.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/init.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/sequential.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/fullConvolution.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/spectral_norm.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/sparseToDense.py -> build/lib.linux-x86_64-3.7/sparseconvnet copying sparseconvnet/unPooling.py -> build/lib.linux-x86_64-3.7/sparseconvnet running build_ext building 'sparseconvnet.SCN' extension creating build/temp.linux-x86_64-3.7 creating build/temp.linux-x86_64-3.7/sparseconvnet creating build/temp.linux-x86_64-3.7/sparseconvnet/SCN :/usr/local/cuda-8.0/bin/nvcc -I/home/deeper/anaconda3/envs/pointpillar/include -I/home/deeper/workspace/SparseConvNet/sparseconvnet/SCN/ -I/home/deeper/anaconda3/envs/pointpillar/lib/python3.7/site-packages/torch/include -I/home/deeper/anaconda3/envs/pointpillar/lib/python3.7/site-packages/torch/include/torch/csrc/api/include -I/home/deeper/anaconda3/envs/pointpillar/lib/python3.7/site-packages/torch/include/TH -I/home/deeper/anaconda3/envs/pointpillar/lib/python3.7/site-packages/torch/include/THC -I:/usr/local/cuda-8.0/include -I/home/deeper/anaconda3/envs/pointpillar/include/python3.7m -c sparseconvnet/SCN/cuda.cu -o build/temp.linux-x86_64-3.7/sparseconvnet/SCN/cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --compiler-options '-fPIC' -std=c++11 -Xcompiler -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=SCN -D_GLIBCXX_USE_CXX11_ABI=0 unable to execute ':/usr/local/cuda-8.0/bin/nvcc': No such file or directory error: command ':/usr/local/cuda-8.0/bin/nvcc' failed with exit status 1 Traceback (most recent call last): File "examples/hello-world.py", line 10, in import sparseconvnet as scn ModuleNotFoundError: No module named 'sparseconvnet'

    it may concerned about "sudo" i guessed. so i try 'sudo python setup.py install " but "sudo python" is usr/bin/python2.7 not the conda python then i try that "sudo /home/deeper/anaconda3/envs/my_env/python setup.py install " Another error occur..

    Traceback (most recent call last): File "setup.py", line 9, in import torch, os File "/home/deeper/anaconda3/envs/pointpillar/lib/python3.7/site-packages/torch/init.py", line 79, in from torch._C import * ImportError: libnvToolsExt.so.1: cannot open shared object file: No such file or directory

    how can I install SparseConvNet correctly? Thank a lot!

    opened by liqisa 6
  • Including sparse convolutions inside traditional model

    Including sparse convolutions inside traditional model

    Hi, first of all thank you so much for sharing this amazing library, I'm willing to use it in my MSc thesis. I have a few questions: If I understood everything correctly, you advise to create a network and forward it an InputBatch created manually. This works well for a network where you have only sparse convolutions. Unfortunately, I have a network where I use a large majority of dense convolutions, and I would like to add some sparse convolutive layers at the end of it. Is it possible to achieve this? I saw that in the pytorch version of the library there is a DenseToSparse layer that would (probably) solve this issue, but you advise not to use it. What is the reason? Could I port it to Lua, or do you plan to implement it in Lua? Thanks in advance for your time, and I'm sorry if I misunderstood something.

    opened by fabvio 6
  • How to get larger values at the output

    How to get larger values at the output

    I am using SparseConvNet for a new task which is different from Segmentation. I need the output as xyz coordinates so my output dimension is 3. My input full_scale=4096. I want my outputs to be between 0 and full_scale. However, my outputs are always between -1 and 1. I am wondering which part of the code is restricting my output from going large and giving me results between 0 and full_scale.

    class Model(nn.Module):
        def __init__(self):
            nn.Module.__init__(self)
            self.sparseModel = scn.Sequential().add(
               scn.InputLayer(data.dimension,data.full_scale, mode=4)).add(
               scn.SubmanifoldConvolution(data.dimension, 3, m, 3, False)).add(  # The 2nd input here is the dimension of the features.
               scn.UNet(data.dimension, block_reps, [m, 2*m, 3*m, 4*m, 5*m, 6*m, 7*m], residual_blocks)).add(
               scn.BatchNormReLU(m)).add(
               scn.OutputLayer(data.dimension))
            self.linear = nn.Linear(m, data.output_dim)
        def forward(self,x):
            x=self.sparseModel(x)
            x=self.linear(x)
            return x
    

    Any help would be appreciated. Thanks

    opened by aniqueakhtar 5
  • RuntimeError: expected scalar type Long but found Float

    RuntimeError: expected scalar type Long but found Float

    I ran examples/hello-world.py and ended up with RuntimeError: expected scalar type Long but found Float. My environment: pytorch=1.13.1, cudatoolkit=10.0.130, gcc/g++=7.5.0. Here is the complete log. How can I fix it?

    (base) root@efbc464e2a2e:/~/SparseConvNet# python examples/hello-world.py 
    wandb: (1) Create a W&B account
    wandb: (2) Use an existing W&B account
    wandb: (3) Don't visualize my results
    wandb: Enter your choice: 3
    wandb: You chose 'Don't visualize my results'
    wandb: Tracking run with wandb version 0.13.7
    wandb: W&B syncing is set to `offline` in this directory.  
    wandb: Run `wandb online` or set WANDB_MODE=online to enable cloud syncing.
    Traceback (most recent call last):
      File "examples/hello-world.py", line 69, in <module>
        output = model(input)
      File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
        return forward_call(*input, **kwargs)
      File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/container.py", line 204, in forward
        input = module(input)
      File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
        return forward_call(*input, **kwargs)
      File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/container.py", line 204, in forward
        input = module(input)
      File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
        return forward_call(*input, **kwargs)
      File "/opt/conda/lib/python3.7/site-packages/sparseconvnet-0.2-py3.7-linux-x86_64.egg/sparseconvnet/maxPooling.py", line 94, in forward
        self.nFeaturesToDrop)
      File "/opt/conda/lib/python3.7/site-packages/sparseconvnet-0.2-py3.7-linux-x86_64.egg/sparseconvnet/maxPooling.py", line 38, in forward
        nFeaturesToDrop)
    RuntimeError: expected scalar type Long but found Float
    wandb: Waiting for W&B process to finish... (failed 1).
    wandb: You can sync this run to the cloud by running:
    wandb: wandb sync /~/SparseConvNet/wandb/offline-run-20221227_173643-737ky2id
    wandb: Find logs at: ./wandb/offline-run-20221227_173643-737ky2id/logs
    
    opened by LeoTheBestCoder 0
  • Building failure related to gcc version

    Building failure related to gcc version

    Hello, I clone this project and use bash build.sh to build in a conda environment but fail. The error message is as below.

    running install
    /home/leo/anaconda3/envs/venv/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    /home/leo/anaconda3/envs/venv/lib/python3.10/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    running bdist_egg
    running egg_info
    creating sparseconvnet.egg-info
    writing sparseconvnet.egg-info/PKG-INFO
    writing dependency_links to sparseconvnet.egg-info/dependency_links.txt
    writing top-level names to sparseconvnet.egg-info/top_level.txt
    writing manifest file 'sparseconvnet.egg-info/SOURCES.txt'
    package init file 'sparseconvnet/SCN/__init__.py' not found (or not a regular file)
    /home/leo/anaconda3/envs/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py:411: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
      warnings.warn(msg.format('we could not find ninja.'))
    reading manifest file 'sparseconvnet.egg-info/SOURCES.txt'
    adding license file 'LICENSE'
    writing manifest file 'sparseconvnet.egg-info/SOURCES.txt'
    installing library code to build/bdist.linux-x86_64/egg
    running install_lib
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.10
    creating build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/activations.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/metadata.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/unPooling.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/averagePooling.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/ioLayers.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/inputBatch.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/spectral_norm.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/maxPooling.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/networkArchitectures.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/submanifoldConvolution.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/randomizedStrideMaxPooling.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/shapeContext.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/convolution.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/dropout.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/randomizedStrideConvolution.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/batchNormalization.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/sparseConvNetTensor.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/sequential.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/deconvolution.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/denseToSparse.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/identity.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/__init__.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/fullConvolution.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/tables.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/utils.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/classificationTrainValidate.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/permutohedralSubmanifoldConvolution.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/sparsify.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/networkInNetwork.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    copying sparseconvnet/sparseToDense.py -> build/lib.linux-x86_64-3.10/sparseconvnet
    running build_ext
    /home/leo/anaconda3/envs/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py:813: UserWarning: The detected CUDA version (10.1) has a minor version mismatch with the version that was used to compile PyTorch (10.2). Most likely this shouldn't be a problem.
      warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
    /home/leo/anaconda3/envs/venv/lib/python3.10/site-packages/torch/utils/cpp_extension.py:820: UserWarning: There are no g++ version bounds defined for CUDA version 10.1
      warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
    building 'sparseconvnet.SCN' extension
    creating build/temp.linux-x86_64-3.10
    creating build/temp.linux-x86_64-3.10/sparseconvnet
    creating build/temp.linux-x86_64-3.10/sparseconvnet/SCN
    /usr/local/cuda/bin/nvcc -I/home/leo/SparseConvNet/sparseconvnet/SCN/ -I/home/leo/anaconda3/envs/venv/lib/python3.10/site-packages/torch/include -I/home/leo/anaconda3/envs/venv/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/home/leo/anaconda3/envs/venv/lib/python3.10/site-packages/torch/include/TH -I/home/leo/anaconda3/envs/venv/lib/python3.10/site-packages/torch/include/THC -I/usr/local/cuda/include -I/home/leo/anaconda3/envs/venv/include/python3.10 -c sparseconvnet/SCN/cuda.cu -o build/temp.linux-x86_64-3.10/sparseconvnet/SCN/cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -std=c++14 -Xcompiler -fopenmp -O3 -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE=\"_gcc\" -DPYBIND11_STDLIB=\"_libstdcpp\" -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" -DTORCH_EXTENSION_NAME=SCN -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_52,code=compute_52 -gencode=arch=compute_52,code=sm_52
    In file included from /usr/local/cuda/include/cuda_runtime.h:83,
                     from <command-line>:
    /usr/local/cuda/include/crt/host_config.h:138:2: error: #error -- unsupported GNU version! gcc versions later than 8 are not supported!
      138 | #error -- unsupported GNU version! gcc versions later than 8 are not supported!
          |  ^~~~~
    error: command '/usr/local/cuda/bin/nvcc' failed with exit code 1
    

    It says that gcc versions later than 8 are not supported. Is there any solution to this problem rather than downgrade gcc version? Thanks for your help!

    opened by LeoTheBestCoder 0
  • RuntimeError: CUDA error: an illegal memory access was encountered

    RuntimeError: CUDA error: an illegal memory access was encountered

    Hi, first of all thanks for sharing this library with all of us! Unfortunately I am encountering few problems while trying to run it. In particular, I tried to build the following network, which is supposed to take as input a sparse tensor of shape (8192, 16384). Part of it is now commented because I tried to locate the origin of the problem, and apparently it happens just with just the first Convolution module (so I commented the rest for now)

    The error that I get is pasted below. The GPU is a quadro gv100, system cuda version 11.4, pytorch 1.11.0 py3.9_cuda11.3_cudnn8.2.0_0

    class HCSparseConvNet1(t.nn.Module):
            def __init__(self, featSize, numOut, size, name = "NN"):
                    super(HCSparseConvNet1, self).__init__()
                    print(size)
                    self.inputLayer = scn.InputLayer(2, size, 2)
                    
                    self.sparseModel = scn.Sequential(scn.Convolution(2,1,4,8,8, True))#, scn.Convolution(2,4,8,8,4, True), scn.LeakyReLU(), scn.Convolution(2,8,16,3,2,True), scn.LeakyReLU(), scn.Convolution(2,16,16, 3,2, True), scn.SparseToDense(2, 16))#, scn.MaxPooling(2,16,8), scn.Convolution(2, 10,10,64,32, False))
                    self.out1 = t.nn.Sequential(t.nn.GroupNorm(1,16), t.nn.Tanh(), t.nn.Conv2d(16,8,3,2), t.nn.GroupNorm(1,8), t.nn.Tanh(), t.nn.Conv2d(8,4,3,1, padding=1), t.nn.GroupNorm(1,4), t.nn.Tanh())
    #self.spatial_size= self.sparseModel.input_spatial_size(size)
                    self.final = t.nn.Sequential(t.nn.Linear(7812, 100), t.nn.LayerNorm(100), t.nn.Tanh(), t.nn.Linear(100, numOut))
    
            def forward(self, x, batchSize):
                    #print(x[0].size(), x[1].size())
                    x = self.inputLayer(x)
                    x = self.sparseModel(x)
                    print(x)
                    #x = self.out1(x)
                    #print(x.size())
                    #x = self.final(x.view(batchSize, -1))
                    return x
    

    The error:

    Traceback (most recent call last):
      File "/home/eddiewrc/galiana2/galianaHCsparseConvNet.py", line 144, in <module>
        sys.exit(main(sys.argv))
      File "/home/eddiewrc/galiana2/galianaHCsparseConvNet.py", line 94, in main
        wrapper.fit(X, Y, device, epochs=50, batch_size = 11, LOG=False)
      File "/home/eddiewrc/galiana2/sources/HCModels.py", line 200, in fit
        yp = self.model.forward([coord, features], batchSize)
      File "/home/eddiewrc/galiana2/sources/HCModels.py", line 58, in forward
        print(x)
      File "/home/eddiewrc/SparseConvNet/sparseconvnet/sparseConvNetTensor.py", line 58, in __repr__
        'features=' + repr(self.features) + \
      File "/home/eddiewrc/miniconda3/lib/python3.9/site-packages/torch/_tensor.py", line 305, in __repr__
        return torch._tensor_str._str(self)
      File "/home/eddiewrc/miniconda3/lib/python3.9/site-packages/torch/_tensor_str.py", line 434, in _str
        return _str_intern(self)
      File "/home/eddiewrc/miniconda3/lib/python3.9/site-packages/torch/_tensor_str.py", line 409, in _str_intern
        tensor_str = _tensor_str(self, indent)
      File "/home/eddiewrc/miniconda3/lib/python3.9/site-packages/torch/_tensor_str.py", line 264, in _tensor_str
        formatter = _Formatter(get_summarized_data(self) if summarize else self)
      File "/home/eddiewrc/miniconda3/lib/python3.9/site-packages/torch/_tensor_str.py", line 296, in get_summarized_data
        return torch.stack([get_summarized_data(x) for x in (start + end)])
    RuntimeError: CUDA error: an illegal memory access was encountered
    CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
    For debugging consider passing CUDA_LAUNCH_BLOCKING=1
    ```.
    
    opened by eddiewrc 2
  • AttributeError: module 'sparseconvnet.SCN' has no attribute 'Metadata_2'

    AttributeError: module 'sparseconvnet.SCN' has no attribute 'Metadata_2'

    Hi, I'm trying to train a Sparse Resnet model using Fastai. However, I'm getting the error of the title:

    AttributeError: module 'sparseconvnet.SCN' has no attribute 'Metadata_2'

    I've tried to follow the source code to try to understand how and when this object is created, I guess it is done on the cpp side, but I can't get it working. Input are feautes & locations of a 2D image, quite similar to the examples.

    opened by emepetres 3
  • Dense to Sparse for input is quite slow

    Dense to Sparse for input is quite slow

    Looking at the samples, it seems that only way to pass a "dense" image to its sparse vectors is by looping over it.

    In my tests, this is really slow to do in python (~15s for a 512x512 image on a RTX2060). Is there a better way to do it?

    Thanks for the great work!

    opened by emepetres 0
Focal Sparse Convolutional Networks for 3D Object Detection (CVPR 2022, Oral)

Focal Sparse Convolutional Networks for 3D Object Detection (CVPR 2022, Oral) This is the official implementation of Focals Conv (CVPR 2022), a new sp

DV Lab 280 Jan 7, 2023
Differentiable Neural Computers, Sparse Access Memory and Sparse Differentiable Neural Computers, for Pytorch

Differentiable Neural Computers and family, for Pytorch Includes: Differentiable Neural Computers (DNC) Sparse Access Memory (SAM) Sparse Differentiab

ixaxaar 302 Dec 14, 2022
CoSMA: Convolutional Semi-Regular Mesh Autoencoder. From Paper "Mesh Convolutional Autoencoder for Semi-Regular Meshes of Different Sizes"

Mesh Convolutional Autoencoder for Semi-Regular Meshes of Different Sizes Implementation of CoSMA: Convolutional Semi-Regular Mesh Autoencoder arXiv p

Fraunhofer SCAI 10 Oct 11, 2022
CondenseNet V2: Sparse Feature Reactivation for Deep Networks

CondenseNetV2 This repository is the official Pytorch implementation for "CondenseNet V2: Sparse Feature Reactivation for Deep Networks" paper by Le Y

Haojun Jiang 74 Dec 12, 2022
Code for our ICASSP 2021 paper: SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

SA-Net: Shuffle Attention for Deep Convolutional Neural Networks (paper) By Qing-Long Zhang and Yu-Bin Yang [State Key Laboratory for Novel Software T

Qing-Long Zhang 199 Jan 8, 2023
《Truly shift-invariant convolutional neural networks》(2021)

Truly shift-invariant convolutional neural networks [Paper] Authors: Anadi Chaman and Ivan Dokmanić Convolutional neural networks were always assumed

Anadi Chaman 46 Dec 19, 2022
《A-CNN: Annularly Convolutional Neural Networks on Point Clouds》(2019)

A-CNN: Annularly Convolutional Neural Networks on Point Clouds Created by Artem Komarichev, Zichun Zhong, Jing Hua from Department of Computer Science

Artёm Komarichev 44 Feb 24, 2022
Spontaneous Facial Micro Expression Recognition using 3D Spatio-Temporal Convolutional Neural Networks

Spontaneous Facial Micro Expression Recognition using 3D Spatio-Temporal Convolutional Neural Networks Abstract Facial expression recognition in video

Bogireddy Sai Prasanna Teja Reddy 103 Dec 29, 2022
This project is a loose implementation of paper "Algorithmic Financial Trading with Deep Convolutional Neural Networks: Time Series to Image Conversion Approach"

Stock Market Buy/Sell/Hold prediction Using convolutional Neural Network This repo is an attempt to implement the research paper titled "Algorithmic F

Asutosh Nayak 136 Dec 28, 2022
Fine-tune pretrained Convolutional Neural Networks with PyTorch

Fine-tune pretrained Convolutional Neural Networks with PyTorch. Features Gives access to the most popular CNN architectures pretrained on ImageNet. A

Alex Parinov 694 Nov 23, 2022
Pytorch implementation of AngularGrad: A New Optimization Technique for Angular Convergence of Convolutional Neural Networks

AngularGrad Optimizer This repository contains the oficial implementation for AngularGrad: A New Optimization Technique for Angular Convergence of Con

mario 124 Sep 16, 2022
Subdivision-based Mesh Convolutional Networks

Subdivision-based Mesh Convolutional Networks The official implementation of SubdivNet in our paper, Subdivion-based Mesh Convolutional Networks Requi

Zheng-Ning Liu 181 Dec 28, 2022
Code for paper: Group-CAM: Group Score-Weighted Visual Explanations for Deep Convolutional Networks

Group-CAM By Zhang, Qinglong and Rao, Lu and Yang, Yubin [State Key Laboratory for Novel Software Technology at Nanjing University] This repo is the o

zhql 98 Nov 16, 2022
Unofficial TensorFlow implementation of Protein Interface Prediction using Graph Convolutional Networks.

[TensorFlow] Protein Interface Prediction using Graph Convolutional Networks Unofficial TensorFlow implementation of Protein Interface Prediction usin

YeongHyeon Park 9 Oct 25, 2022
Generating Anime Images by Implementing Deep Convolutional Generative Adversarial Networks paper

AnimeGAN - Deep Convolutional Generative Adverserial Network PyTorch implementation of DCGAN introduced in the paper: Unsupervised Representation Lear

Rohit Kukreja 23 Jul 21, 2022
PyTorch implementation of "ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context" (INTERSPEECH 2020)

ContextNet ContextNet has CNN-RNN-transducer architecture and features a fully convolutional encoder that incorporates global context information into

Sangchun Ha 24 Nov 24, 2022
PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021.

GCResNet PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021. The code will

null 11 May 19, 2022
A PyTorch implementation of " EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks."

EfficientNet A PyTorch implementation of EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. [arxiv] [Official TF Repo] Implemen

AhnDW 298 Dec 10, 2022
Official PyTorch Implementation of Convolutional Hough Matching Networks, CVPR 2021 (oral)

Convolutional Hough Matching Networks This is the implementation of the paper "Convolutional Hough Matching Network" by J. Min and M. Cho. Implemented

Juhong Min 70 Nov 22, 2022