Single/multi view image(s) to voxel reconstruction using a recurrent neural network

Related tags

Deep Learning 3D-R2N2
Overview

3D-R2N2: 3D Recurrent Reconstruction Neural Network

This repository contains the source codes for the paper Choy et al., 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction, ECCV 2016. Given one or multiple views of an object, the network generates voxelized ( a voxel is the 3D equivalent of a pixel) reconstruction of the object in 3D.

Citing this work

If you find this work useful in your research, please consider citing:

@inproceedings{choy20163d,
  title={3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction},
  author={Choy, Christopher B and Xu, Danfei and Gwak, JunYoung and Chen, Kevin and Savarese, Silvio},
  booktitle = {Proceedings of the European Conference on Computer Vision ({ECCV})},
  year={2016}
}

News

  • [2020-01-25] Using a dense ocupancy grid for 3D reconstruction requires a large amount of memory and computation. We present a new auto-diff library for sparse tensors that can reconstruct objects in high resolution. Please refer to the 3D sparsity pattern reconstruction page for 3D reconstruction using a sparse tensor.

Project Page

The project page is available at http://cvgl.stanford.edu/3d-r2n2/.

Overview

Overview Left: images found on Ebay, Amazon, Right: overview of 3D-R2N2

Traditionally, single view reconstruction and multi-view reconstruction are disjoint problems that have been dealt using different approaches. In this work, we first propose a unified framework for both single and multi-view reconstruction using a 3D Recurrent Reconstruction Neural Network (3D-R2N2).

3D-Convolutional LSTM 3D-Convolutional GRU Inputs (red cells + feature) for each cell (purple)
3D-LSTM 3D-GRU 3D-LSTM

We can feed in images in random order since the network is trained to be invariant to the order. The critical component that enables the network to be invariant to the order is the 3D-Convolutional LSTM which we first proposed in this work. The 3D-Convolutional LSTM selectively updates parts that are visible and keeps the parts that are self-occluded.

Networks We used two different types of networks for the experiments: a shallow network (top) and a deep residual network (bottom).

Results

Please visit the result visualization page to view 3D reconstruction results interactively.

Datasets

We used ShapeNet models to generate rendered images and voxelized models which are available below (you can follow the installation instruction below to extract it to the default directory).

Installation

The package requires python3. You can follow the direction below to install virtual environment within the repository or install anaconda for python 3.

  • Download the repository
git clone https://github.com/chrischoy/3D-R2N2.git

cd 3D-R2N2
conda create -n py3-theano python=3.6
source activate py3-theano
conda install pygpu
pip install -r requirements.txt
  • copy the theanorc file to the $HOME directory
cp .theanorc ~/.theanorc

Running demo.py

  • Install meshlab (skip if you have another mesh viewer). If you skip this step, demo code will not visualize the final prediction.
sudo apt-get install meshlab
  • Run the demo code and save the final 3D reconstruction to a mesh file named prediction.obj
python demo.py prediction.obj

The demo code takes 3 images of the same chair and generates the following reconstruction.

Image 1 Image 2 Image 3 Reconstruction
  • Deactivate your environment when you are done
deactivate

Training the network

  • Activate the virtual environment before you run the experiments.
source py3/bin/activate
  • Download datasets and place them in a folder named ShapeNet
mkdir ShapeNet/
wget http://cvgl.stanford.edu/data2/ShapeNetRendering.tgz
wget http://cvgl.stanford.edu/data2/ShapeNetVox32.tgz
tar -xzf ShapeNetRendering.tgz -C ShapeNet/
tar -xzf ShapeNetVox32.tgz -C ShapeNet/
  • Train and test the network using the training shell script
./experiments/script/res_gru_net.sh

Note: The initial compilation might take awhile if you run the theano for the first time due to various compilations. The problem will not persist for the subsequent runs.

Using cuDNN

To use cuDNN library, you have to download cuDNN from the nvidia website. Then, extract the files to any directory and append the directory to the environment variables like the following. Please replace the /path/to/cuDNN/ to the directory that you extracted cuDNN.

export LD_LIBRARY_PATH=/path/to/cuDNN/lib64:$LD_LIBRARY_PATH
export CPATH=/path/to/cuDNN/include:$CPATH
export LIBRARY_PATH=/path/to/cuDNN/lib64:$LD_LIBRARY_PATH

For more details, please refer to http://deeplearning.net/software/theano/library/sandbox/cuda/dnn.html

Follow-up Paper

Gwak et al., Weakly supervised 3D Reconstruction with Adversarial Constraint, project website

Supervised 3D reconstruction has witnessed a significant progress through the use of deep neural networks. However, this increase in performance requires large scale annotations of 2D/3D data. In this paper, we explore inexpensive 2D supervision as an alternative for expensive 3D CAD annotation. Specifically, we use foreground masks as weak supervision through a raytrace pooling layer that enables perspective projection and backpropagation. Additionally, since the 3D reconstruction from masks is an ill posed problem, we propose to constrain the 3D reconstruction to the manifold of unlabeled realistic 3D shapes that match mask observations. We demonstrate that learning a log-barrier solution to this constrained optimization problem resembles the GAN objective, enabling the use of existing tools for training GANs. We evaluate and analyze the manifold constrained reconstruction on various datasets for single and multi-view reconstruction of both synthetic and real images.

License

MIT License

Comments
  • Execution Errors on Custom Images (both same and different size)

    Execution Errors on Custom Images (both same and different size)

    While experimenting a little bit I faced this exception after about 1 minute of execution on there custom 3 images (resolution 640 x 480), I am on Gentoo system with Nvidia 8 stack, latest theano stack 0.9.0:

    zangetsu@ares ~/proj/neural-networks/3D-R2N2 $ python demo.py prediction.obj 
Using gpu device 0: GeForce GTX 960M (CNMeM is disabled, cuDNN 5110)
/usr/lib64/python3.5/site-packages/theano/sandbox/cuda/__init__.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5.
  warnings.warn(warn)
loading network parameters from output/ResidualGRUNet/default_model/weights.npy
Compiling testing function
Traceback (most recent call last):
  File "theano/scan_module/scan_perform.pyx", line 397, in theano.scan_module.scan_perform.perform (/home/zangetsu/.theano/compiledir_Linux-4.11-gentoo-x86_64-Intel-R-_Core-TM-_i7-6700HQ_CPU_@_2.60GHz-with-gentoo-2.3-Intel_R_Core_TM_i7-6700HQ_CPU_@_2.60GHz-3.5.3-64/scan_perform/mod.cpp:4193)
ValueError: CudaNdarray_CopyFromCudaNdarray: need same dimensions for dim 1, destination=3, source=4

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.5/site-packages/theano/compile/function_module.py", line 859, in __call__
    outputs = self.fn()
  File "/usr/lib64/python3.5/site-packages/theano/scan_module/scan_op.py", line 951, in rval
    r = p(n, [x[0] for x in i], o)
  File "/usr/lib64/python3.5/site-packages/theano/scan_module/scan_op.py", line 940, in <lambda>
    self, node)
  File "theano/scan_module/scan_perform.pyx", line 405, in theano.scan_module.scan_perform.perform (/home/zangetsu/.theano/compiledir_Linux-4.11-gentoo-x86_64-Intel-R-_Core-TM-_i7-6700HQ_CPU_@_2.60GHz-with-gentoo-2.3-Intel_R_Core_TM_i7-6700HQ_CPU_@_2.60GHz-3.5.3-64/scan_perform/mod.cpp:4316)
  File "/usr/lib64/python3.5/site-packages/theano/gof/link.py", line 314, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/usr/lib64/python3.5/site-packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "theano/scan_module/scan_perform.pyx", line 397, in theano.scan_module.scan_perform.perform (/home/zangetsu/.theano/compiledir_Linux-4.11-gentoo-x86_64-Intel-R-_Core-TM-_i7-6700HQ_CPU_@_2.60GHz-with-gentoo-2.3-Intel_R_Core_TM_i7-6700HQ_CPU_@_2.60GHz-3.5.3-64/scan_perform/mod.cpp:4193)
ValueError: CudaNdarray_CopyFromCudaNdarray: need same dimensions for dim 1, destination=3, source=4
Apply node that caused the error: GpuIncSubtensor{Set;::, ::, int64:int64:, int64:int64:}(CudaNdarrayConstant{[[[[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]]]}, Rebroadcast{1}.0, Constant{3}, Constant{130}, Constant{3}, Constant{130})
Toposort index: 116
Inputs types: [CudaNdarrayType(float32, (True, False, False, False)), CudaNdarrayType(float32, (True, False, False, False)), Scalar(int64), Scalar(int64), Scalar(int64), Scalar(int64)]
Inputs shapes: [(1, 3, 133, 133), (1, 4, 480, 640), (), (), (), ()]
Inputs strides: [(0, 17689, 133, 1), (0, 307200, 640, 1), (), (), (), ()]
Inputs values: ['not shown', 'not shown', 3, 130, 3, 130]
Outputs clients: [[GpuContiguous(GpuIncSubtensor{Set;::, ::, int64:int64:, int64:int64:}.0)]]

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "demo.py", line 84, in <module>
    main()
  File "demo.py", line 66, in main
    voxel_prediction, _ = solver.test_output(demo_imgs)
  File "/home/zangetsu/proj/neural-networks/3D-R2N2/lib/solver.py", line 232, in test_output
    results = self._test_output(x, y_val)
  File "/usr/lib64/python3.5/site-packages/theano/compile/function_module.py", line 871, in __call__
    storage_map=getattr(self.fn, 'storage_map', None))
  File "/usr/lib64/python3.5/site-packages/theano/gof/link.py", line 314, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/usr/lib64/python3.5/site-packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib64/python3.5/site-packages/theano/compile/function_module.py", line 859, in __call__
    outputs = self.fn()
  File "/usr/lib64/python3.5/site-packages/theano/scan_module/scan_op.py", line 951, in rval
    r = p(n, [x[0] for x in i], o)
  File "/usr/lib64/python3.5/site-packages/theano/scan_module/scan_op.py", line 940, in <lambda>
    self, node)
  File "theano/scan_module/scan_perform.pyx", line 405, in theano.scan_module.scan_perform.perform (/home/zangetsu/.theano/compiledir_Linux-4.11-gentoo-x86_64-Intel-R-_Core-TM-_i7-6700HQ_CPU_@_2.60GHz-with-gentoo-2.3-Intel_R_Core_TM_i7-6700HQ_CPU_@_2.60GHz-3.5.3-64/scan_perform/mod.cpp:4316)
  File "/usr/lib64/python3.5/site-packages/theano/gof/link.py", line 314, in raise_with_op
    reraise(exc_type, exc_value, exc_trace)
  File "/usr/lib64/python3.5/site-packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "theano/scan_module/scan_perform.pyx", line 397, in theano.scan_module.scan_perform.perform (/home/zangetsu/.theano/compiledir_Linux-4.11-gentoo-x86_64-Intel-R-_Core-TM-_i7-6700HQ_CPU_@_2.60GHz-with-gentoo-2.3-Intel_R_Core_TM_i7-6700HQ_CPU_@_2.60GHz-3.5.3-64/scan_perform/mod.cpp:4193)
ValueError: CudaNdarray_CopyFromCudaNdarray: need same dimensions for dim 1, destination=3, source=4
Apply node that caused the error: GpuIncSubtensor{Set;::, ::, int64:int64:, int64:int64:}(CudaNdarrayConstant{[[[[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]

  [[ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   ..., 
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]
   [ 0.  0.  0. ...,  0.  0.  0.]]]]}, Rebroadcast{1}.0, Constant{3}, Constant{130}, Constant{3}, Constant{130})
Toposort index: 116
Inputs types: [CudaNdarrayType(float32, (True, False, False, False)), CudaNdarrayType(float32, (True, False, False, False)), Scalar(int64), Scalar(int64), Scalar(int64), Scalar(int64)]
Inputs shapes: [(1, 3, 133, 133), (1, 4, 480, 640), (), (), (), ()]
Inputs strides: [(0, 17689, 133, 1), (0, 307200, 640, 1), (), (), (), ()]
Inputs values: ['not shown', 'not shown', 3, 130, 3, 130]
Outputs clients: [[GpuContiguous(GpuIncSubtensor{Set;::, ::, int64:int64:, int64:int64:}.0)]]

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
Apply node that caused the error: forall_inplace,gpu,scan_fn}(Shape_i{0}.0, GpuSubtensor{int64:int64:int8}.0, GpuIncSubtensor{InplaceSet;:int64:}.0, GpuIncSubtensor{InplaceSet;:int64:}.0, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, 4D)>, <CudaNdarrayType(float32, matrix)>, <CudaNdarrayType(float32, matrix)>, <CudaNdarrayType(float32, matrix)>, <CudaNdarrayType(float32, matrix)>, GpuDimShuffle{x,x,0,x,x}.0, GpuDimShuffle{x,0}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,0,x,x}.0, GpuDimShuffle{x,x,0,x,x}.0, GpuDimShuffle{x,x,0,x,x}.0, Shape_i{4}.0, Shape_i{3}.0, Shape_i{1}.0, Shape_i{0}.0, Shape_i{4}.0, Shape_i{3}.0, Shape_i{1}.0, Shape_i{0}.0, Shape_i{4}.0, Shape_i{3}.0, Shape_i{1}.0, Shape_i{0}.0, GpuReshape{4}.0, GpuReshape{4}.0, GpuReshape{4}.0)
Toposort index: 348
Inputs types: [TensorType(int64, scalar), CudaNdarrayType(float32, 5D), CudaNdarrayType(float32, (False, True, False, False, False, False)), CudaNdarrayType(float32, (False, True, False, False, False, False)), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, matrix), CudaNdarrayType(float32, matrix), CudaNdarrayType(float32, matrix), CudaNdarrayType(float32, matrix), CudaNdarrayType(float32, (True, True, False, True, True)), CudaNdarrayType(float32, row), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, False, True, True)), CudaNdarrayType(float32, (True, True, False, True, True)), CudaNdarrayType(float32, (True, True, False, True, True)), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), TensorType(int64, scalar), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D)]
Inputs shapes: [(), (3, 1, 4, 480, 640), (2, 1, 4, 128, 4, 4), (3, 1, 4, 128, 4, 4), (96, 3, 7, 7), (96, 96, 3, 3), (128, 96, 1, 1), (128, 96, 3, 3), (128, 128, 3, 3), (256, 128, 1, 1), (256, 128, 3, 3), (256, 256, 3, 3), (256, 256, 3, 3), (256, 256, 3, 3), (256, 256, 1, 1), (256, 256, 3, 3), (256, 256, 3, 3), (256, 256, 3, 3), (256, 256, 3, 3), (2304, 1024), (1024, 8192), (1024, 8192), (1024, 8192), (1, 1, 128, 1, 1), (1, 1024), (1, 256, 1, 1), (1, 256, 1, 1), (1, 256, 1, 1), (1, 256, 1, 1), (1, 256, 1, 1), (1, 256, 1, 1), (1, 256, 1, 1), (1, 256, 1, 1), (1, 128, 1, 1), (1, 128, 1, 1), (1, 96, 1, 1), (1, 96, 1, 1), (1, 128, 1, 1), (1, 256, 1, 1), (1, 256, 1, 1), (1, 1, 128, 1, 1), (1, 1, 128, 1, 1), (), (), (), (), (), (), (), (), (), (), (), (), (384, 128, 3, 3), (384, 128, 3, 3), (384, 128, 3, 3)]
Inputs strides: [(), (1228800, 0, 307200, 640, 1), (8192, 0, 2048, 16, 4, 1), (8192, 0, 2048, 16, 4, 1), (147, 49, 7, 1), (864, 9, 3, 1), (96, 1, 0, 0), (864, 9, 3, 1), (1152, 9, 3, 1), (128, 1, 0, 0), (1152, 9, 3, 1), (2304, 9, 3, 1), (2304, 9, 3, 1), (2304, 9, 3, 1), (256, 1, 0, 0), (2304, 9, 3, 1), (2304, 9, 3, 1), (2304, 9, 3, 1), (2304, 9, 3, 1), (1024, 1), (8192, 1), (8192, 1), (8192, 1), (0, 0, 1, 0, 0), (0, 1), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0, 0), (0, 0, 1, 0, 0), (), (), (), (), (), (), (), (), (), (), (), (), (1152, 9, 3, 1), (1152, 9, 3, 1), (1152, 9, 3, 1)]
Inputs values: [array(3), 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', 'not shown', array(3), array(3), array(3), array(128), array(3), array(3), array(3), array(128), array(3), array(3), array(3), array(128), 'not shown', 'not shown', 'not shown']
Outputs clients: [[GpuSubtensor{int64}(forall_inplace,gpu,scan_fn}.0, ScalarFromTensor.0)], [GpuSubtensor{int64:int64:int8}(forall_inplace,gpu,scan_fn}.1, ScalarFromTensor.0, ScalarFromTensor.0, Constant{1})]]

HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.

    Here is full trace as per HINTs: https://pastebin.com/yh2eBg81

    I converted images to 127x127 but that didn't help.

    opened by archenroot 8
  • After activating env, runing demo.py cause

    After activating env, runing demo.py cause "Problem occurred during compilation"

    Hi, i was trying to reproduce the experiment, and i follow the given instructions, however, i will get an environment with python 3.4 where the code complain syntax error. I try to explicitly define a python 3.5 environment, however, more errors comes, listed as following, could any one help me? Thanks!

    Problem occurred during compilation with the command line below:

    - /usr/bin/g++ -shared -g -march=core-avx2 -mcx16 -msahf -mmovbe -maes -mpclmul -mpopcnt -mabm -mno-lwp -mfma -mno-fma4 -mno-xop -mbmi -mbmi2 -mno-tbm -mavx -mavx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=15360 -mtune=generic -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/root/.pyenv/versions/3.5.1/lib/python3.5/site-packages/numpy/core/include -I/root/.pyenv/versions/3.5.1/include/python3.5m -I/root/.pyenv/versions/3.5.1/lib/python3.5/site-packages/theano/gof -fvisibility=hidden -o /root/.theano/compiledir_Linux-4.4--generic-x86_64-with-debian-jessie-sid-x86_64-3.5.1-64/lazylinker_ext/lazylinker_ext.so /root/.theano/compiledir_Linux-4.4--generic-x86_64-with-debian-jessie-sid-x86_64-3.5.1-64/lazylinker_ext/mod.cpp -L/root/.pyenv/versions/3.5.1/lib -lpython3.5m
- /usr/bin/ld: /root/.pyenv/versions/3.5.1/lib/libpython3.5m.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
- /root/.pyenv/versions/3.5.1/lib/libpython3.5m.a: error adding symbols: Bad value
- collect2: error: ld returned 1 exit status
- 
- Traceback (most recent call last):
-   File "/root/.pyenv/versions/3.5.1/lib/python3.5/site-packages/theano/gof/lazylinker_c.py", line 74, in <module>
-     raise ImportError()
- ImportError
- 
- During handling of the above exception, another exception occurred:
- 
- Traceback (most recent call last):
-   File "/root/.pyenv/versions/3.5.1/lib/python3.5/site-packages/theano/gof/lazylinker_c.py", line 91, in <module>
-     raise ImportError()
- ImportError
- 
- During handling of the above exception, another exception occurred:
- 
- Traceback (most recent call last):
-   File "demo.py", line 17, in <module>
-     from models import load_model
-   File "/home/yht/yang3d/hope/3D-R2N2/models/__init__.py", line 1, in <module>
-     from models.gru_net import GRUNet
-   File "/home/yht/yang3d/hope/3D-R2N2/models/gru_net.py", line 4, in <module>
-     import theano
-   File "/root/.pyenv/versions/3.5.1/lib/python3.5/site-packages/theano/__init__.py", line 63, in <module>
-     from theano.compile import (
-   File "/root/.pyenv/versions/3.5.1/lib/python3.5/site-packages/theano/compile/__init__.py", line 9, in <module>
-     from theano.compile.function_module import *
-   File "/root/.pyenv/versions/3.5.1/lib/python3.5/site-packages/theano/compile/function_module.py", line 22, in <module>
-     import theano.compile.mode
-   File "/root/.pyenv/versions/3.5.1/lib/python3.5/site-packages/theano/compile/mode.py", line 12, in <module>
-     import theano.gof.vm
-   File "/root/.pyenv/versions/3.5.1/lib/python3.5/site-packages/theano/gof/vm.py", line 638, in <module>
-     from . import lazylinker_c
-   File "/root/.pyenv/versions/3.5.1/lib/python3.5/site-packages/theano/gof/lazylinker_c.py", line 126, in <module>
-     preargs=args)
-   File "/root/.pyenv/versions/3.5.1/lib/python3.5/site-packages/theano/gof/cmodule.py", line 2204, in compile_str
-     (status, compile_stderr.replace('\n', '. ')))
- Exception: Compilation failed (return status=1): /usr/bin/ld: /root/.pyenv/versions/3.5.1/lib/libpython3.5m.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC. /root/.pyenv/versions/3.5.1/lib/libpython3.5m.a: error adding symbols: Bad value. collect2: error: ld returned 1 exit status.
    invalid 
    opened by ghost 8
  • No such file or directory: 'curl'

    No such file or directory: 'curl'

    Sorry to bother you, I'm new to this, when I run the demo.py, I got this problem, may you please tell me how to fix it ?thank you

    Using gpu device 0: GeForce GTX 1050 Ti (CNMeM is disabled, cuDNN 6021) /home/tony/.local/lib/python3.5/site-packages/theano/sandbox/cuda/init.py:600: UserWarning: Your cuDNN version is more recent than the one Theano officially supports. If you see any problems, try updating Theano or downgrading cuDNN to version 5. warnings.warn(warn) Downloading a pretrained model Traceback (most recent call last): File "demo.py", line 84, in main() File "demo.py", line 55, in main download_model(DEFAULT_WEIGHTS) File "demo.py", line 34, in download_model '--create-dirs', '-o', fn]) File "/usr/lib/python3.5/subprocess.py", line 557, in call with Popen(*popenargs, **kwargs) as p: File "/usr/lib/python3.5/subprocess.py", line 947, in init restore_signals, start_new_session) File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child raise child_exception_type(errno_num, err_msg) FileNotFoundError: [Errno 2] No such file or directory: 'curl'

    opened by Tony1Q84 7
  • Datasets and Pre-trained model is not available

    Datasets and Pre-trained model is not available

    The links for datasets and pre-trained model are put within the domain of Standford University. Outside people without login username and password cannot access them. Could you please put them in open places?

    opened by SummitChen 6
  • nvcc return status 2

    nvcc return status 2

    I also got another error message on another machine: Exception: ('The following error happened while compiling the node', GpuDnnConvDesc{border_mode='valid', subsample=(1, 1), conv_mode='conv', precision='float32'}(TensorConstant{[816 32 34 34]}, MakeVector{dtype='int64'}.0), '\n', 'nvcc return status', 2, 'for cmd', 'nvcc -shared -O3 -arch=sm_61 -m64 -Xcompiler -fno-math-errno,-Wno-unused-label,-Wno-unused-variable,-Wno-write-strings,-DCUDA_NDARRAY_CUH=mc72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden -Xlinker -rpath,/home/hzxie/.theano/compiledir_Linux-4.10--generic-x86_64-with-Ubuntu-16.04-xenial-x86_64-3.5.2-64/cuda_ndarray -I/usr/local/lib/python3.5/dist-packages/theano/sandbox/cuda -I/opt/cuda/include -I/usr/local/lib/python3.5/dist-packages/numpy/core/include -I/usr/include/python3.5m -I/usr/local/lib/python3.5/dist-packages/theano/gof -o /home/hzxie/.theano/compiledir_Linux-4.10--generic-x86_64-with-Ubuntu-16.04-xenial-x86_64-3.5.2-64/tmpdwc9ej0h/mce178ed9c0cdab911c3a009d42a487fa.so mod.cu -L/usr/lib -lcudnn -lpython3.5m -lcudart', "[GpuDnnConvDesc{border_mode='valid', subsample=(1, 1), conv_mode='conv', precision='float32'}(TensorConstant{[816 32 34 34]}, <TensorType(int64, vector)>)]")

    Can you tell me how to solve it? Thank you very much.

    opened by hzxie 3
  • ResidualGRUNet TEST

    ResidualGRUNet TEST

    Hi, I use Ubuntu 16.04 to implement the project and get a good result. And ,When I training the network with the function train_net(), it get a normal result. But ,When I test the network with the function test_net(),I get all of results of mAP are 0. Is it right, and should I change something in your scripts? Thanks for your answer！

    opened by hangongzi 3
  • compilation terminated

    compilation terminated

    @jgwak @liujiaxing Hi,I was trying to reproduce this experiment,and I follow the given instructions. My compilation environment is Python3.5.3. However,I meet the same problem that Liujiaxing met . Could you give me some help or any suggestions? Thanks very much.

    `Problem occurred during compilation with the command line below: /usr/bin/g++ -shared -g -march=ivybridge -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mno-movbe -maes -mno-sha -mpclmul -mpopcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mavx -mno-avx2 -msse4.2 -msse4.1 -mno-lzcnt -mno-rtm -mno-hle -mrdrnd -mf16c -mfsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mxsave -mxsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 -mno-clflushopt -mno-xsavec -mno-xsaves -mno-avx512dq -mno-avx512bw -mno-avx512vl -mno-avx512ifma -mno-avx512vbmi -mno-clwb -mno-mwaitx -mno-clzero -mno-pku --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=3072 -mtune=ivybridge -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m64 -fPIC -I/home/wj/3D-R2N2/py3/lib/python3.5/site-packages/numpy/core/include -I/usr/include/python3.5m -I/home/wj/3D-R2N2/py3/include/python3.5m -I/home/wj/3D-R2N2/py3/lib/python3.5/site-packages/theano/gof -L/usr/lib -fvisibility=hidden -o /home/wj/.theano/compiledir_Linux-4.10--generic-x86_64-with-Ubuntu-17.04-zesty-x86_64-3.5.3-64/lazylinker_ext/lazylinker_ext.so /home/wj/.theano/compiledir_Linux-4.10--generic-x86_64-with-Ubuntu-17.04-zesty-x86_64-3.5.3-64/lazylinker_ext/mod.cpp -lpython3.5m /home/wj/.theano/compiledir_Linux-4.10--generic-x86_64-with-Ubuntu-17.04-zesty-x86_64-3.5.3-64/lazylinker_ext/mod.cpp:1:20: fatal error: Python.h: No such file or directory #include <Python.h> ^ compilation terminated.

    Traceback (most recent call last): File "/home/wj/3D-R2N2/py3/lib/python3.5/site-packages/theano/gof/lazylinker_c.py", line 75, in raise ImportError() ImportError

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "/home/wj/3D-R2N2/py3/lib/python3.5/site-packages/theano/gof/lazylinker_c.py", line 92, in raise ImportError() ImportError

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "demo.py", line 17, in from models import load_model File "/home/wj/3D-R2N2/models/init.py", line 1, in from models.gru_net import GRUNet File "/home/wj/3D-R2N2/models/gru_net.py", line 4, in import theano File "/home/wj/3D-R2N2/py3/lib/python3.5/site-packages/theano/init.py", line 66, in from theano.compile import ( File "/home/wj/3D-R2N2/py3/lib/python3.5/site-packages/theano/compile/init.py", line 10, in from theano.compile.function_module import * File "/home/wj/3D-R2N2/py3/lib/python3.5/site-packages/theano/compile/function_module.py", line 21, in import theano.compile.mode File "/home/wj/3D-R2N2/py3/lib/python3.5/site-packages/theano/compile/mode.py", line 10, in import theano.gof.vm File "/home/wj/3D-R2N2/py3/lib/python3.5/site-packages/theano/gof/vm.py", line 662, in from . import lazylinker_c File "/home/wj/3D-R2N2/py3/lib/python3.5/site-packages/theano/gof/lazylinker_c.py", line 127, in preargs=args) File "/home/wj/3D-R2N2/py3/lib/python3.5/site-packages/theano/gof/cmodule.py", line 2316, in compile_str (status, compile_stderr.replace('\n', '. '))) Exception: Compilation failed (return status=1): /home/wj/.theano/compiledir_Linux-4.10--generic-x86_64-with-Ubuntu-17.04-zesty-x86_64-3.5.3-64/lazylinker_ext/mod.cpp:1:20: fatal error: Python.h: No such file or directory. #include <Python.h>. `

    opened by GenaWang 3
  • TypeError: Outputs must be theano Variable or Out instances. Received [Subtensor{int64::}.0] of type <class 'list'>

    TypeError: Outputs must be theano Variable or Out instances. Received [Subtensor{int64::}.0] of type

    Hi,

    I am meeting the following problem, and whatever i try to use the different versions beyond python-3.5.1 like you said before in the last issue , or run by the version of PYTHON_CONFIGURE_OPTS="--enable-shared", it still doesn't work :(

    Traceback (most recent call last):
  File "demo.py", line 84, in <module>
    main()
  File "demo.py", line 66, in main
    voxel_prediction, _ = solver.test_output(demo_imgs)
  File "/home/wenyangming/3D-R2N2/lib/solver.py", line 218, in test_output
    self.net.activations])
  File "/home/wenyangming/.pyenv/versions/env3_5_1/lib/python3.5/site-packages/theano/compile/function.py", line 320, in function
    output_keys=output_keys)
  File "/home/wenyangming/.pyenv/versions/env3_5_1/lib/python3.5/site-packages/theano/compile/pfunc.py", line 442, in pfunc
    no_default_updates=no_default_updates)
  File "/home/wenyangming/.pyenv/versions/env3_5_1/lib/python3.5/site-packages/theano/compile/pfunc.py", line 227, in rebuild_collect_shared
    ' of type ' + str(type(v)))
TypeError: Outputs must be theano Variable or Out instances. Received [Subtensor{int64::}.0] of type <class 'list'>
****

    Did you have some suggestions for this problem? Thanks a lot and looking forward to your response.

    opened by alanyannick 3
  • Assertion Error

    Assertion Error

    Using cuDNN version 6021 on context None Mapped name None to device cuda0: Tesla K80 (0000:00:04.0) /home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/skimage/transform/_warps.py:84: UserWarning: The default mode, 'constant', will be changed to 'reflect' in skimage 0.15. warn("The default mode, 'constant', will be changed to 'reflect' in " /home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/tensor/nnet/conv.py:98: UserWarning: theano.tensor.nnet.conv.conv2d is deprecated. Use theano.tensor.nnet.conv2d instead. warnings.warn("theano.tensor.nnet.conv.conv2d is deprecated." /mnt/disks/ext/3D-R2N2/lib/layers.py:354: UserWarning: DEPRECATION: the 'ds' parameter is not going to exist anymore as it is going to be replaced by the parameter 'ws'. padding=self._padding) /mnt/disks/ext/3D-R2N2/lib/layers.py:354: UserWarning: DEPRECATION: the 'padding' parameter is not going to exist anymore as it is going to be replaced by the parameter 'pad'. padding=self._padding) Traceback (most recent call last): File "demo.py", line 89, in main() File "demo.py", line 71, in main voxel_prediction, _ = solver.test_output(demo_imgs)
    File "/mnt/disks/ext/3D-R2N2/lib/solver.py", line 220, in test_output *self.net.activations]) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/compile/function.py", line 317, in function output_keys=output_keys) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/compile/pfunc.py", line 486, in pfunc output_keys=output_keys) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/compile/function_module.py", line 1841, in orig_function fn = m.create(defaults) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/compile/function_module.py", line 1715, in create input_storage=input_storage_lists, storage_map=storage_map) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/gof/link.py", line 699, in make_thunk storage_map=storage_map)[:3] File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/gof/vm.py", line 1084, in make_all impl=impl)) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/scan_module/scan_op.py", line 866, in make_thunk on_unused_input='ignore') File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/compile/function.py", line 317, in function output_keys=output_keys) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/compile/pfunc.py", line 486, in pfunc output_keys=output_keys) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/compile/function_module.py", line 1839, in orig_function name=name) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/compile/function_module.py", line 1519, in init optimizer_profile = optimizer(fgraph) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/gof/opt.py", line 99, in call return self.optimize(fgraph) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/gof/opt.py", line 88, in optimize ret = self.apply(fgraph, *args, **kwargs) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/gof/opt.py", line 242, in apply sub_prof = optimizer.optimize(fgraph) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/gof/opt.py", line 88, in optimize ret = self.apply(fgraph, *args, **kwargs) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/gof/opt.py", line 242, in apply sub_prof = optimizer.optimize(fgraph) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/gof/opt.py", line 88, in optimize ret = self.apply(fgraph, *args, **kwargs) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/gpuarray/opt.py", line 417, in apply node.outputs) File "/home/imran_salam_24/anaconda3/envs/my_env/lib/python3.6/site-packages/theano/gpuarray/opt.py", line 1496, in local_gpua_error_convop """

    AssertionError: ('The following error happened while compiling the node', forall_inplace,gpu,scan_fn}(Shape_i{0}.0, GpuSubtensor{int64:int64:int8}.0, GpuIncSubtensor{InplaceSet;:int64:}.0, GpuIncSubtensor{InplaceSet;:int64:}.0, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, 4D)>, <GpuArrayType(float32, matrix)>, <GpuArrayType(float32, matrix)>, <GpuArrayType(float32, matrix)>, <GpuArrayType(float32, matrix)>, InplaceGpuDimShuffle{x,x,0,x,x}.0, InplaceGpuDimShuffle{x,0}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, InplaceGpuDimShuffle{x,0,x,x}.0, GpuFromHost.0, InplaceGpuDimShuffle{x,x,0,x,x}.0, GpuFromHost.0, GpuFromHost.0, GpuFromHost.0, GpuFromHost.0, GpuFromHost.0, GpuReshape{4}.0, GpuFromHost.0, GpuFromHost.0, GpuFromHost.0, GpuFromHost.0, GpuReshape{4}.0, InplaceGpuDimShuffle{x,x,0,x,x}.0, GpuFromHost.0, GpuFromHost.0, GpuFromHost.0, GpuFromHost.0, GpuFromHost.0, GpuReshape{4}.0), '\n', '\nConvOp does not work with the gpuarray backend.\n\nUse the new convolution interface to have GPU convolution working:\ntheano.tensor.nnet.conv2d()\n')

    My theanorc file is ; [global] floatX = float32 device = cuda0

    opened by imransalam 2
  • final output dimension

    final output dimension

    Hi,

    I found out in your code and your paper as well that the final output of the network is [batch_size, n_vox, 2, n_vox, n_vox]. Why would you need 2 output channel?

    Cheers

    opened by likojack 2
  • Optional Surface Only View Parameter

    Optional Surface Only View Parameter

    Added an option in voxel2obj which by default only saves the voxels found the the model's surface . This speeds up viewing in MeshLab considerably for large models.

    opened by EdwardSmith1884 1
  • Weights file not available

    Weights file not available

    Hello and thanks for your code!

    I've tried to download weights file, but, unfortunately, when i'm clicking http://cvgl.stanford.edu/data2/ResidualGRUNet.npy new tab is opened and loading of this new tab is never ends. And download never starts. using wget doesn't help either.

    opened by DaddyWesker 2
  • How to find out label of image category label from the ID given

    How to find out label of image category label from the ID given

    Hi,

    I was wondering if there's any straightforward way to find the category label (for eg 'airplane') for the images each of the 2D renderings based on the ID given?

    opened by HiokHian 0
  • Training on Pascal Dataset

    Training on Pascal Dataset

    Hi

    I was browsing through the visualizations that you have shown and tried to reimplement the same ones myself but I was not able to do so. The model that you have trained is on shapenet. Should I be able retrain a model on Pascal or a custom dataset in order to get similar results to your pascal visualizations?

    opened by mahnoor-fatima-saad 0
Owner
Chris Choy
Research Scientist @NVIDIA. Previously Ph.D. from Stanford Vision and Learning Lab @StanfordVL (SVL), Stanford AI Lab, SAIL.
Chris Choy
GitHub
Implementation of Bidirectional Recurrent Independent Mechanisms (Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules)

BRIMs Bidirectional Recurrent Independent Mechanisms Implementation of the paper Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neura

Sarthak Mittal 26 May 26, 2022
Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal, multi-exposure and multi-focus image fusion.

U2Fusion Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal (VIS-IR, medical), multi

Han Xu 129 Dec 11, 2022
[ICCV'21] UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction

UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction Project Page | Paper | Supplementary | Video This reposit

null 331 Dec 28, 2022
Compute descriptors for 3D point cloud registration using a multi scale sparse voxel architecture

MS-SVConv : 3D Point Cloud Registration with Multi-Scale Architecture and Self-supervised Fine-tuning Compute features for 3D point cloud registration

null 42 Jul 25, 2022
Toward Realistic Single-View 3D Object Reconstruction with Unsupervised Learning from Multiple Images (ICCV 2021)

Table of Content Introduction Getting Started Datasets Installation Experiments Training & Testing Pretrained models Texture fine-tuning Demo Toward R

VinAI Research 42 Dec 5, 2022
Code for "Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency" paper

UNICORN ?? Webpage | Paper | BibTex PyTorch implementation of "Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency" pap

null 118 Jan 6, 2023
OcclusionFusion: realtime dynamic 3D reconstruction based on single-view RGB-D

OcclusionFusion (CVPR'2022) Project Page | Paper | Video Overview This repository contains the code for the CVPR 2022 paper OcclusionFusion, where we

Wenbin Lin 193 Dec 15, 2022
CoReNet is a technique for joint multi-object 3D reconstruction from a single RGB image.

CoReNet CoReNet is a technique for joint multi-object 3D reconstruction from a single RGB image. It produces coherent reconstructions, where all objec

Google Research 80 Dec 25, 2022
Blender add-on: Add to Cameras menu: View → Camera, View → Add Camera, Camera → View, Previous Camera, Next Camera

Blender add-on: Camera additions In 3D view, it adds these actions to the View|Cameras menu: View → Camera : set the current camera to the 3D view Vie

German Bauer 11 Feb 8, 2022
"MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction" (CVPRW 2022) & (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)

MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction (CVPRW 2022) Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Z

Yuanhao Cai 274 Jan 5, 2023
Implementation of ICCV2021(Oral) paper - VMNet: Voxel-Mesh Network for Geodesic-aware 3D Semantic Segmentation

VMNet: Voxel-Mesh Network for Geodesic-Aware 3D Semantic Segmentation Created by Zeyu HU Introduction This work is based on our paper VMNet: Voxel-Mes

HU Zeyu 82 Dec 27, 2022
Official implementation of "SinIR: Efficient General Image Manipulation with Single Image Reconstruction" (ICML 2021)

SinIR (Official Implementation) Requirements To install requirements: pip install -r requirements.txt We used Python 3.7.4 and f-strings which are in

null 47 Oct 11, 2022
Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set (CVPRW 2019). A PyTorch implementation.

Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set —— PyTorch implementation This is an unofficial offici

Sicheng Xu 833 Dec 28, 2022
Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistency[ECCV 2020]

Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistency(ECCV 2020) This is an official python implementati

null 304 Jan 3, 2023
Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.

Non-Rigid Neural Radiance Fields This is the official repository for the project "Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synt

Facebook Research 296 Dec 29, 2022
OHLC Average Prediction of Apple Inc. Using LSTM Recurrent Neural Network

Stock Price Prediction of Apple Inc. Using Recurrent Neural Network OHLC Average Prediction of Apple Inc. Using LSTM Recurrent Neural Network Dataset:

Nouroz Rahman 410 Jan 5, 2023
Speech Separation Using an Asynchronous Fully Recurrent Convolutional Neural Network

Speech Separation Using an Asynchronous Fully Recurrent Convolutional Neural Network This repository is the official implementation of Speech Separati

Kai Li (李凯) 116 Nov 9, 2022
PyTorch implementation of Hierarchical Multi-label Text Classification: An Attention-based Recurrent Network

hierarchical-multi-label-text-classification-pytorch Hierarchical Multi-label Text Classification: An Attention-based Recurrent Network Approach This

Mingu Kang 17 Dec 13, 2022
This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods from a single image under variations in viewing angle, lighting, and common occlusions.

NoW Evaluation This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard e

Soubhik Sanyal 71 Dec 30, 2022