Neural Scene Flow Prior
Xueqian Li, Jhony Kaesemodel Pontes, Simon Lucey
Will appear on Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS, 2021) as spotlight.
arXiv link: https://arxiv.org/pdf/2111.01253.pdf
-
Scene flow results on Argoverse
-
Point cloud integration (left: densified point cloud; right: sparse point cloud)
Prerequisites
This code is based on PyTorch implementation, and tested on torch=1.6.0 with CUDA 10.1 OR torch=1.7.1 with CUDA 10.2.
For a detailed installation guide, please go to requirements.txt.
Dataset
We provide four datasets we used in our paper. You may download datasets used in the paper from these anonymous links:
-
KITTI (266MB)
-
Argoverse (5.3GB)
-
nuScenes (436MB)
-
FlyingThings3D (436MB)
After you download the dataset, you can create a symbolic link in the ./dataset folder as ./dataset/kitti, ./dataset/argoverse, ./dataset/nuscenes, and ./dataset/flyingthings.
Optimization
Since we use neural scene flow prior for runtime optimization, our method does not include any "training".
Just run following lines for a simple optimization on a small KITTI Scene Flow dataset (only 50 testing samples)
python optimization.py \
--dataset KITTISceneFlowDataset \
--dataset_path dataset/kitti \
--exp_name KITTI_2048_points \
--batch_size 1 \
--iters 5000 \
--compute_metrics \
--num_points 2048 \
--hidden_units 128 \
--lr 0.008 \
--backward_flow \
--early_patience 70 \
--visualize
You can then play with these configurations. We provide commands we used to generate results in the small point coud (2048 points) experiments and large point cloud (all points included) experiments.
1. small point cloud (2048 points)
KITTI Scene Flow
python optimization.py \
--dataset KITTISceneFlowDataset \
--dataset_path dataset/kitti \
--exp_name KITTI_2048_points \
--batch_size 1 \
--iters 5000 \
--compute_metrics \
--num_points 2048 \
--hidden_units 128 \
--lr 0.008 \
--backward_flow \
--early_patience 70 \
--visualize
Argoverse Scene Flow
python optimization.py \
--dataset ArgoverseSceneFlowDataset \
--dataset_path dataset/argoverse \
--exp_name Argoverse_2048_points \
--batch_size 1 \
--iters 5000 \
--compute_metrics \
--num_points 2048 \
--hidden_units 128 \
--lr 0.008 \
--backward_flow \
--early_patience 30 \
--visualize
nuScenes Scene Flow
python optimization.py \
--dataset NuScenesSceneFlowDataset \
--dataset_path dataset/nuscenes \
--exp_name Argoverse_2048_points \
--batch_size 1 \
--iters 5000 \
--compute_metrics \
--num_points 2048 \
--hidden_units 128 \
--lr 0.008 \
--backward_flow \
--early_patience 30 \
--visualize
FlyingThings3D
python optimization.py \
--dataset FlyingThings3D \
--dataset_path dataset/flyingthings \
--exp_name FlyingThings_2048_points \
--batch_size 1 \
--iters 5000 \
--compute_metrics \
--num_points 2048 \
--hidden_units 128 \
--lr 0.008 \
--backward_flow \
--early_patience 30 \
--visualize
2. dense point cloud (all points included)
KITTI Scene Flow
python optimization.py \
--dataset KITTISceneFlowDataset \
--dataset_path dataset/kitti \
--exp_name KITTI_2048_points \
--batch_size 1 \
--iters 5000 \
--compute_metrics \
--use_all_points \
--hidden_units 128 \
--lr 0.001 \
--early_patience 100 \
--visualize
Argoverse Scene Flow
python optimization.py \
--dataset ArgoverseSceneFlowDataset \
--dataset_path dataset/argoverse \
--exp_name Argoverse_2048_points \
--batch_size 1 \
--iters 5000 \
--compute_metrics \
--use_all_points \
--hidden_units 128 \
--lr 0.003 \
--backward_flow \
--early_patience 100 \
--visualize
Contributing
If you find the project useful for your research, you may cite,
@article{li2021neural,
title={Neural scene flow prior},
author={Li, Xueqian and Pontes, Jhony Kaesemodel and Lucey, Simon},
journal={arXiv preprint arXiv:2111.01253},
year={2021}
}
585 Jan 4, 2023
49 Jul 27, 2022
139 Dec 14, 2022
77 Dec 16, 2022
88 Jan 4, 2023
151 Dec 26, 2022
178 Dec 21, 2022
85 Dec 22, 2022
43 Dec 5, 2022
90 Oct 19, 2022
182 Dec 30, 2022
66 Nov 16, 2022
124 Dec 27, 2022
28 Nov 16, 2022
55 Dec 3, 2022
33 Dec 12, 2022
35 Nov 20, 2022
52 Dec 22, 2022
259 Dec 28, 2022