When using Cuda 11 our model returns the following error:

File "/home/zgojcic/anaconda3/envs/rigid_3dsf/lib/python3.7/site-packages/MinkowskiEngine-0.5.1-py3.7-linux-x86_64.egg/MinkowskiEngine/MinkowskiConvolution.py", line 84, in forward
    coordinate_manager._manager,
RuntimeError: /home/zgojcic/Documents/Rigid3DSceneFlow/MinkowskiEngine/src/convolution_gpu.cu:85, assertion (in_feat.size(0) == p_map_manager->size(in_key)) failed. Invalid in_feat size 0 != 5296

It seems that this is due to the combination of Cuda 11 with MinkowskiEngine. The issue is currently under investigation https://github.com/NVIDIA/MinkowskiEngine/issues/330

Until solved we suggest using Cuda 10.2 or 10.1.

Hi, when I run python train.py ./configs/train/train_weakly_supervised.yaml to train the network from scratch using our dataset, my system memory usage will slowly increase until it max out the system memory and then the traning will crash. I have 16gb of system memory and the training can only go on for a little more than one epoch with ~16000 training samples. I tried to lower the num_workers to 4 and lower the batch size to 2 but they didn't seem to resolve the issue.

Hi! Thank you for your wonderful work. I tried downloading the checkpoint by the script you provided. However, it seems the link saving the models expired, and I cannot find the pre-trained models. Could you please tell me how I can get them?

Thank you very much for your kind help!

Hi @zgojcic thanks for sharing this inspiring work!

Since the output of DBSCAN is unordered and even the number of clusters might be different, so how did you determine the corresponding FG instances to compute the rigid transformation? And will the instance segmentation result be good enough for DBSCAN at the beginning of the training stage?

Another small question is why you use different tau for Eq.9 and Eq.10? Is the reason that they are doing softmax among a different numbers of correspondences?

Best, Xuyang

Hello! I want to change the model_best.pt file into .onnx file, however, I don't clearly find out the input parameter of network. It will help a lot if you can answer me the input parameter of the network. If you can offer the .onnx file of the model will help me a lot.

Thank you!

Hi, thanks for sharing. I wonder why you remove the points invisible within the front camera for semanticKITTI. Based on your paper, it doesn't seem necessary.Is it any difference when I use full point cloud to test or train?

Hi,

I saw the closed issue regarding this topic, but for me was not clear. Where I can find the points that need to be converted? Can you provide please the code for doing this, before using the Keyshot?

Hi, thanks for the nice work.

When computing the rotation matrix after SVD, the following line transposes both V and U: https://github.com/zgojcic/Rigid3DSceneFlow/blob/7fa57e3ddccf605dca63ded04825bba2272cae4a/lib/utils.py#L335 But I think only U should be transposed here. Is this an error?

I find another implementation in the source code: https://github.com/zgojcic/Rigid3DSceneFlow/blob/7fa57e3ddccf605dca63ded04825bba2272cae4a/lib/model/minkowski/MinkowskiFlow.py#L344 Here only U is transposed.

Hi, thank you for the great work.

As you have mentioned in the paper, the coordinate systems for kitti and for waymo are different. In order to account for the difference, you said you have "transform[ed] these points into a cordinate system centered at the location of the LiDAR sensor in the KITTI setup". I wonder if you had applied any transformation on the ego motion data of waymo-open dataset(i.e. R and t) as well, since the direction of the ego vehicle is along z-axis for the kitti dataset and along x-axis for the waymo dataset. Also, the ego transformation does not necessarily have to be "aligned" with the absolute coordinate frame. If I blindly apply the transformation (that I have used for the transformation of the point clouds) to the ego coordinate, the absolute R is not aligned with the world coordinate; in other words, R is quite different from the identity matrix, I. However, the estimated R matrix as well as the GT R matrix for kitti dataset are "almost aligned with the world coordinate"; namely, they are very close to identity. Have you applied any other tricks to bring the R back to I in the training process? Could you explain a bit more on how you have translated the ego motion from waymo to kitti? Thank you in advance.

Hi, thanks for the nice work. I currently want to perform a test on semanticKITTI. How is the 'pose' in the npz file defined and obtained? For example, I downloaded the complete tar archive, and load the data in 11/000000_000001.npz, for the frame_000000, the data['pose_s'] is an identity matrix, corresponding to the global coordinate system, and data['pose_t'] is same as the line-1 of pose.txt provided by semantickitti. Is this pose in the Camera coordinate system or the LiDAR coordinate system? Because I saw in the supplemental material that the LIDAR data should be projected to the Camera coordinate system. Is the pose here also in the camera coordinate system?

I have tried several ways, but I can’t convert the two frames of point clouds to one coordinate system through data['pose_t'] (In the former example, data['pose_s'] is the identity matrix). Can you give me some hints?

This is a wonder work! I have a question regarding how is the dataset such as semantic_kitti processed? When I try to print out what is in one of the npz files in semantic_kitti dataset I get this: ['pc1', 'pc2', 'sem_label_s', 'sem_label_t', 'inst_label_s', 'inst_label_t', 'mot_label_s', 'mot_label_t', 'pose_s', 'pose_t'], I wonder what each of these is and how to get them. Thanks!