Neural Scene Graphs for Dynamic Scene (CVPR 2021)

Overview

Neural Scene Graphs for Dynamic Scene (CVPR 2021)

alt text

Project Page | Paper

Julian Ost, Fahim Mannan, Nils Thuerey, Julian Knodt, Felix Heide

Implementation of Neural Scene Graphs, that optimizes multiple radiance fields to represent different objects and a static scene background. Learned representations can be rendered with novel object compositions and views.

Original repository forked from the Implementation of "NeRF: Neural Radiance Fields" by Mildenhall et al.: Original NeRF Implementation, original readme


Getting started

The whole script is currently optimized for the usage with Virtual KITTI 2 Dataset and KITTI

Quick Start

Train a Virtual KITTI 2 Scene

conda create -n neural_scene_graphs --file requirements.txt -c conda-forge -c menpo
conda activate neural_scene_graphs
cd neural-scene-graphs
bash download_virtual_kitti.sh
python main.py --config example_configs/config_vkitti2_Scene06.py
tensorboard --logdir=example_weights/summaries --port=6006

Render a pretrained KITTI Scene from a trained Scene Graph Models

Follow the instructions under data preparation to setup the KITTI dataset.

conda create -n neural_scene_graphs --file requirements.txt -c conda-forge -c menpo
conda activate neural_scene_graphs
cd neural-scene-graphs
bash download_weights_kitti.sh
python main.py --config example_configs/config_kitti_0006_example_render.py
tensorboard --logdir=example_weights/summaries --port=6006

Disclaimer: The codebase is optimized to run on larger GPU servers with a lot of free CPU memory. To test on local and low memory,

  1. Use chunk and netchunk in the config files to limit parallel computed rays and sampling points.

or

  1. resize and retrain with
--training_factor = 'downsampling factor'

or change to the desired factor in your config file.


Data Preperation

KITTI

  1. Get the KITTI MOT dataset, from which you need:
    1. Left color images
    2. Right color images
    3. GPS/IMU data
    4. Camera Calibration Files
    5. Training labels
  2. Extract everything to ./data/kitti and keep the data structure
  3. Neural Scene Graphs is well tested and published on real front-facing scenarios with only small movements along the camera viewing direction. We therefore prepared selected config files for KITTI Scenes (0001, 0002, 0006)

Virtual KITTI 2

bash ./download_virtual_kitti.sh

Training

To optimize models on a subsequence of Virtual KITTI 2 or KITTI, create the environment, download the data set (1.2) and optimize the (pre-trained) background and object models together:

conda create -n neural_scene_graphs --file requirements.txt -c conda-forge -c menpo
conda activate neural_scene_graphs

vkitti2 example:

python main.py --config example_configs/config_vkitti2_Scene06.txt
tensorboard --logdir=example_weights/summaries --port=6006

KITTI example:

python main.py --config example_configs/config_kitti_0006_example_train.txt
tensorboard --logdir=example_weights/summaries --port=6006

Rendering a Sequence

Render a pretrained KITTI sequence

bash download_weights_kitti.sh
python main.py --config example_configs/config_kitti_0006_example_render.txt

To render a pre-trained download the weights or use your own model.

bash download_weights_kitti.sh

To make a full render pass over all selected images (between the first and last frame) run the provided config with 'render_only=True'.

  • To render only the outputs of the static background node use 'bckg_only=True'
  • for all dynamic parts set 'obj_only=True' & 'white_bkgd=True'
python main.py --config example_configs/config_kitti_0006_example_render.txt

Citation

@InProceedings{Ost_2021_CVPR,
    author    = {Ost, Julian and Mannan, Fahim and Thuerey, Nils and Knodt, Julian and Heide, Felix},
    title     = {Neural Scene Graphs for Dynamic Scenes},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {2856-2865}
}
Comments
  • Getting Blas GEMM launch erorr when trying to repro the initial virtual kitti code

    Getting Blas GEMM launch erorr when trying to repro the initial virtual kitti code

    python main.py --config example_configs/config_vkitti2_Scene06.txt

    Loading poses from: ./data/vkitti2/Scene06/clone/pose.txt Loading bbox from: ./data/vkitti2/Scene06/clone/bbox.txt Loading info from: ./data/vkitti2/Scene06/clone/info.txt /home/j316chuck/dev/nerf/neural-scene-graphs/data_loader/load_vkitti.py:308: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray bboxes = np.array(bboxes)[selected_range] Loaded vkitti (6, 375, 1242, 3) [375, 1242, 725.0087] ./data/vkitti2/Scene06/clone [5.] in this scene [6.] in this scene [12.] in this scene [4.] in this scene [9.] in this scene [10.] in this scene 2022-07-05 01:43:27.951776: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2022-07-05 01:43:27.981609: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-07-05 01:43:27.981708: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: NVIDIA GeForce RTX 3090 major: 8 minor: 6 memoryClockRate(GHz): 1.8 pciBusID: 0000:01:00.0 2022-07-05 01:43:27.982037: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0 2022-07-05 01:43:27.983233: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0 2022-07-05 01:43:27.984156: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0 2022-07-05 01:43:27.985026: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0 2022-07-05 01:43:27.986967: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0 2022-07-05 01:43:27.987981: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0 2022-07-05 01:43:27.991330: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2022-07-05 01:43:27.991421: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-07-05 01:43:27.991541: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-07-05 01:43:27.991592: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0 2022-07-05 01:43:27.992007: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA 2022-07-05 01:43:28.016309: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3600000000 Hz 2022-07-05 01:43:28.016706: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x382e820 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2022-07-05 01:43:28.016828: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2022-07-05 01:43:28.088636: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-07-05 01:43:28.088792: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x38014c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2022-07-05 01:43:28.088806: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 3090, Compute Capability 8.6 2022-07-05 01:43:28.088915: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-07-05 01:43:28.088981: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: NVIDIA GeForce RTX 3090 major: 8 minor: 6 memoryClockRate(GHz): 1.8 pciBusID: 0000:01:00.0 2022-07-05 01:43:28.089010: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0 2022-07-05 01:43:28.089017: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0 2022-07-05 01:43:28.089022: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0 2022-07-05 01:43:28.089028: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0 2022-07-05 01:43:28.089033: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0 2022-07-05 01:43:28.089039: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0 2022-07-05 01:43:28.089044: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2022-07-05 01:43:28.089071: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-07-05 01:43:28.089137: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-07-05 01:43:28.089184: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0 2022-07-05 01:43:28.089205: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0 2022-07-05 01:43:28.089330: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix: 2022-07-05 01:43:28.089336: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0 2022-07-05 01:43:28.089342: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N 2022-07-05 01:43:28.089382: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-07-05 01:43:28.089452: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2022-07-05 01:43:28.089505: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0. 2022-07-05 01:43:28.089523: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21873 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:01:00.0, compute capability: 8.6) MODEL 63 27 <class 'int'> <class 'int'> True (?, 90) (?, 63) (?, 27) MODEL 319 54 <class 'int'> <class 'int'> True (?, 373) (?, 319) (?, 54) MODEL 319 54 <class 'int'> <class 'int'> True (?, 373) (?, 319) (?, 54) Found ckpts [] get rays done, concats adding object nodes to each ray (2794500, 15, 3) Using Ray Object Node intersections Removing object None WARNING:tensorflow:From /home/j316chuck/dev/nerf/neural-scene-graphs/neural_scene_graph_helper.py:510: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.where in 2.0, which has the same broadcast rule as np.where 5 is hit by 15285.0 rays! This is 6 times less than the most hit object! Adding Car support Factor 1.0 /home/j316chuck/dev/nerf/neural-scene-graphs/prepare_input_helper.py:437: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray _eq_sz_rays = np.repeat(np.concatenate(np.array(_new_rays_rgb[_id_hit]), axis=0), _hit_factor, axis=0) 6 is hit by 57126.0 rays! This is 1 times less than the most hit object! Adding Car support Factor 1.0 12 is hit by 5044.0 rays! This is 18 times less than the most hit object! Adding Car support Factor 1.0 4 is hit by 4320.0 rays! This is 22 times less than the most hit object! Adding Car support Factor 1.0 9 is hit by 13278.0 rays! This is 7 times less than the most hit object! Adding Truck and Van support Factor 13.0 10 is hit by 95678.0 rays! This is 1 times less than the most hit object! Adding Car support Factor 1.0 Adding dense sampling close to objects. (3074703, 15, 3) shuffle rays done Begin TRAIN views are [0 1 2 3 4 5] TEST views are [1, 5] VAL views are [1, 5] WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see:

    • https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
    • https://github.com/tensorflow/addons
    • https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue.

    2022-07-05 01:46:12.119972: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0 2022-07-05 01:46:40.414762: E tensorflow/stream_executor/cuda/cuda_blas.cc:428] failed to run cuBLAS routine: CUBLAS_STATUS_EXECUTION_FAILED Traceback (most recent call last): File "main.py", line 1859, in train() File "main.py", line 1738, in train verbose=i < 10, retraw=True, **render_kwargs_train) File "main.py", line 696, in render all_ret = batchify_rays(rays, chunk, **kwargs) File "main.py", line 595, in batchify_rays ret = render_rays(rays_flat[i:i+chunk], **kwargs) File "main.py", line 450, in render_rays raw_bckg = network_query_fn(pts, viewdirs, network_fn) File "main.py", line 945, in network_query_fn netchunk=args.netchunk) File "main.py", line 60, in run_network outputs_flat = batchify(fn, netchunk)(embedded) File "main.py", line 27, in ret return tf.concat([fn(inputs[i:i+chunk]) for i in range(0, inputs.shape[0], chunk)], 0) File "main.py", line 27, in return tf.concat([fn(inputs[i:i+chunk]) for i in range(0, inputs.shape[0], chunk)], 0) File "/home/j316chuck/anaconda3/envs/neural_scene_graphs/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 898, in call outputs = self.call(cast_inputs, *args, **kwargs) File "/home/j316chuck/anaconda3/envs/neural_scene_graphs/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/network.py", line 695, in call return self._run_internal_graph(inputs, training=training, mask=mask) File "/home/j316chuck/anaconda3/envs/neural_scene_graphs/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/network.py", line 844, in _run_internal_graph output_tensors = layer(computed_tensors, **kwargs) File "/home/j316chuck/anaconda3/envs/neural_scene_graphs/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 898, in call outputs = self.call(cast_inputs, *args, **kwargs) File "/home/j316chuck/anaconda3/envs/neural_scene_graphs/lib/python3.6/site-packages/tensorflow_core/python/keras/layers/core.py", line 1050, in call outputs = gen_math_ops.mat_mul(inputs, self.kernel) File "/home/j316chuck/anaconda3/envs/neural_scene_graphs/lib/python3.6/site-packages/tensorflow_core/python/ops/gen_math_ops.py", line 6126, in mat_mul _six.raise_from(_core._status_to_exception(e.code, message), None) File "", line 3, in raise_from tensorflow.python.framework.errors_impl.InternalError: Blas GEMM launch failed : a.shape=(49152, 63), b.shape=(63, 256), m=49152, n=256, k=63 [Op:MatMul]

    opened by chucktang98 4
  • Black Background during training

    Black Background during training

    With certain train/test splits on Scenes 1 and 2 (using the settings in https://github.com/princeton-computational-imaging/neural-scene-graphs/issues/7) I see either a black background or nothing at all:

    image

    Any guidance on what could be causing this would be greatly appreciated!

    opened by hturki 1
  • training problem - PSNR converges at about 13

    training problem - PSNR converges at about 13

    hi, great work! During training, I find that for several scenes such as scene 3,4,7, their PSNR can only achieve about 13. If use pretrained examples_6, the PSNR of Scene6 can achieve about 21(still lower than 23 in the paper) origin_img_v2_ae3d4b6c-c9c0-4de7-a9a7-1111d61690eg I wonder what is the reason? Are there any training tricks? Below is my conifg of Scene7, which is modefied from examples of Scene6: config_kitti_0007_example_train.txt

    Looking forward to your reply Thanks

    opened by AddASecond 1
  • use shadows

    use shadows

    Thanks for your great work and sharing your code to reconstruct the dynamic scene.

    I have seen the paper mentioning that the NSG can learn the shadow implicitly, and saw there is use_shadows in the arguments. However, this argument seems not to be used anywhere in the code base. Is this because the release version did not have such functionality?

    Best, Kaicheng

    opened by kcyu2014 1
  • The validation set is the subset of training set?

    The validation set is the subset of training set?

    The code here shows that the training set contains all frames in the scene, and the validation (test) set is the part of the training set. Is this setting reasonable for a novel view synthesis task?

    opened by ootts 1
  • Bounding Box or instance Seg?

    Bounding Box or instance Seg?

    Hi, I really appreciate your amazing work.

    After reading your code and demo, I found there are still some traces in demo, objects are rendered in a box style.

    I also check your code, you have tested the instance segmentation in early experiments. I am really interested in the reason why you choose bounding box instead of instance segmentations.

    Best

    opened by xingweiqu 0
  • Long  video demo

    Long video demo

    Hi, I have trained a model on Virtual KITTI 2, following the training pipeline from readme. However, when I use the trained model for inference, I set "render_only" to True and only get three frames of inference results. So I would like to ask you how I can get a long video like the one shown in your project page based on this code. Looking forward to your reply, thank you!

    opened by fupiao1998 0
  • Replicating paper evaluation results

    Replicating paper evaluation results

    Thanks for the paper and making your code public. I had some questions around reproducing the results in Table 1 of the paper:

    • To confirm, is the training sequence is some set of input frames from a KITTI sequence (frames 65 to 120 for Scene 06) and then are the evaluation numbers reported by rendering all of the training set frames (ie: is the validation set identical to the training set?) or some subset
    • Is the evaluation script present in this repo? Running python main.py --config example_configs/config_kitti_0006_example_render.txt seems to render 3 frames + a video of those 3 frames, but I'm not sure that it's showing any evaluation metrics.
    • Are the numbers reported against subsequences of scenes 0001, 0002, and 0006? If so, would it be possible to learn the starting and ending frames used for 0001 and 0002? The README mentions config files for those scenes but I don't see them in the repo
    opened by hturki 0
  • can i run it on windows conda environment?

    can i run it on windows conda environment?

    "PackagesNotFoundError: The following packages are not available from current channels:

    • imagemagick" There is no windows version of imagemagic in conda channels "https://conda.anaconda.org/conda-forge/" I can find Win64 install package in website http://www.imagemagick.org/script/download.php but how to use it in the project?
    opened by xudan79 0
  • Can not render only the outputs of the static background node

    Can not render only the outputs of the static background node

    Hi,

    I run the example in the Rendering a Sequence with 'bckg_only=True'. Also, I test the code for all dynamic parts with 'obj_only=True' & 'white_bkgd=True'. However, both two cases have a error. ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 0, the array at index 0 has size 112 and the array at index 1 has size [98.]

    This is because in the main.py line 1518. rays_rgb = np.concatenate([rays, images[:, None, ...]], 1) The size of rays is (112, 2, 375, 1242, 3) and the size of images is (98, 1, 375, 1242, 3). They can not concatenate together. How to solve this problem?

    Thank you for any suggestion. Letian

    opened by letian-zhang 0
  • The problem of realizing the pre training model

    The problem of realizing the pre training model

    Hello! Follow your steps to install the environment and relevant data. The experimental equipment and environment meet the requirements. According to the pre training model, only three steps are performed to generate a 500K vido that cannot be opened mp4。 Prompt the following warning, It's where I didn't set it? image BoZhang

    opened by thefoxsayold 2
Owner
null
The code for the CVPR 2021 paper Neural Deformation Graphs, a novel approach for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.

Neural Deformation Graphs Project Page | Paper | Video Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction Aljaž Božič, Pablo P

Aljaz Bozic 134 Dec 16, 2022
Implementation for the EMNLP 2021 paper "Interactive Machine Comprehension with Dynamic Knowledge Graphs".

Interactive Machine Comprehension with Dynamic Knowledge Graphs Implementation for the EMNLP 2021 paper. Dependencies apt-get -y update apt-get instal

Xingdi (Eric) Yuan 19 Aug 23, 2022
Code for CVPR 2021 oral paper "Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts"

Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts The rapid progress in 3D scene understanding has come with growing dem

Facebook Research 182 Dec 30, 2022
Deep learning with dynamic computation graphs in TensorFlow

TensorFlow Fold TensorFlow Fold is a library for creating TensorFlow models that consume structured data, where the structure of the computation graph

null 1.8k Dec 28, 2022
A Pytorch implement of paper "Anomaly detection in dynamic graphs via transformer" (TADDY).

TADDY: Anomaly detection in dynamic graphs via transformer This repo covers an reference implementation for the paper "Anomaly detection in dynamic gr

Yue Tan 21 Nov 24, 2022
Code accompanying "Dynamic Neural Relational Inference" from CVPR 2020

Code accompanying "Dynamic Neural Relational Inference" This codebase accompanies the paper "Dynamic Neural Relational Inference" from CVPR 2020. This

Colin Graber 48 Dec 23, 2022
Dynamic View Synthesis from Dynamic Monocular Video

Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer This repository contains code to compute depth from a

Intelligent Systems Lab Org 2.3k Jan 1, 2023
Dynamic View Synthesis from Dynamic Monocular Video

Dynamic View Synthesis from Dynamic Monocular Video Project Website | Video | Paper Dynamic View Synthesis from Dynamic Monocular Video Chen Gao, Ayus

Chen Gao 139 Dec 28, 2022
Dynamic vae - Dynamic VAE algorithm is used for anomaly detection of battery data

Dynamic VAE frame Automatic feature extraction can be achieved by probability di

null 10 Oct 7, 2022
(CVPR 2021) PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds

PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds by Mutian Xu*, Runyu Ding*, Hengshuang Zhao, and Xiaojuan Qi. Int

CVMI Lab 228 Dec 25, 2022
Dynamic Slimmable Network (CVPR 2021, Oral)

Dynamic Slimmable Network (DS-Net) This repository contains PyTorch code of our paper: Dynamic Slimmable Network (CVPR 2021 Oral). Architecture of DS-

Changlin Li 197 Dec 9, 2022
git git《Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking》(CVPR 2021) GitHub:git2] 《Masksembles for Uncertainty Estimation》(CVPR 2021) GitHub:git3]

Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking Ning Wang, Wengang Zhou, Jie Wang, and Houqiang Li Accepted by CVPR

NingWang 236 Dec 22, 2022
Fine-grained Control of Image Caption Generation with Abstract Scene Graphs

Faster R-CNN pretrained on VisualGenome This repository modifies maskrcnn-benchmark for object detection and attribute prediction on VisualGenome data

Shizhe Chen 7 Apr 20, 2021
Official PyTorch implementation of "Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets" (ICLR 2021)

Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets This is the official PyTorch implementation for the paper Rapid Neural A

null 48 Dec 26, 2022
Official PyTorch code of DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization (ICCV 2021 Oral).

DeepPanoContext (DPC) [Project Page (with interactive results)][Paper] DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context G

Cheng Zhang 66 Nov 16, 2022
Code for "Neural 3D Scene Reconstruction with the Manhattan-world Assumption" CVPR 2022 Oral

News 05/10/2022 To make the comparison on ScanNet easier, we provide all quantitative and qualitative results of baselines here, including COLMAP, COL

ZJU3DV 365 Dec 30, 2022
Code for "Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks", CVPR 2021

Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks This repository contains the code that accompanies our CVPR 20

Despoina Paschalidou 161 Dec 20, 2022
Seeing Dynamic Scene in the Dark: High-Quality Video Dataset with Mechatronic Alignment (ICCV2021)

Seeing Dynamic Scene in the Dark: High-Quality Video Dataset with Mechatronic Alignment This is a pytorch project for the paper Seeing Dynamic Scene i

DV Lab 21 Nov 28, 2022
Official PyTorch code of Holistic 3D Scene Understanding from a Single Image with Implicit Representation (CVPR 2021)

Implicit3DUnderstanding (Im3D) [Project Page] Holistic 3D Scene Understanding from a Single Image with Implicit Representation Cheng Zhang, Zhaopeng C

Cheng Zhang 149 Jan 8, 2023