Hello, I'm sorry to disturb you. According to the paper, I only konw about you did it under the GeForce RTX 3090. But some errors occurred when I wanted to reproduce your experiment. So could you please tell me exactly what's your experiment environment? (Something like requirements.txt )

Thanks for your amazing work, and I'm care about the time of training consuming. From the config_server_kitti.yaml, look like use 4x3090 GPU with batch size 8. Would you like to share your training time? More, limited by hardware，do you think that can get comparable performance to the paper report using a single 3090? In addition, is it possible to use amp in torch？

Hello, thanks you for sharing the code.

I have run the pmf model in semantic kitti training set using the default settings. However, the evaluation result in validation set is lower than the value 63.9 reported in the paper.

Meanwhile, I also run the salsanext model provied in the codebase in semantic kitti training set using the default settings. The evaluation result in validation set is higer than the value 59.4 reported in the paper and the pmf model.

Could you give some suggestions?

when i run PMF code on Sensat Urban Dataset，it show that “UnboundLocalError: local variable 'mean_acc' referenced before assignment” and "UnboundLocalError: local variable 'lr' referenced before assignment"。How can I solve this problem？ Complete error reporting information is as follows

/home/yczhou/anaconda3/envs/buct-bishe/lib/python3.7/site-packages/torch/distributed/launch.py:186: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See 
https://pytorch.org/docs/stable/distributed.html#launch-utility for 
further instructions

  FutureWarning,
WARNING:torch.distributed.run:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
*****************************************
| distributed init (rank 1): env://
| distributed init (rank 2): env://
| distributed init (rank 0): env://
>> Init a recoder at  ../../../experiments/PMF-sensat/log_SensatUrban_PMFNet-resnet101_bs12-lr0.001_baseline-timestamp
loading data frame...
0it [00:00, ?it/s]
Using 0 data frame from train split
loading data frame...
0it [00:00, ?it/s]
Using 0 data frame from val split
Generate 0 samples from train split
Generate 0 samples from val split
loading data frame...
0it [00:00, ?it/s]
Using 0 data frame from train split
loading data frame...
0it [00:00, ?it/s]
Using 0 data frame from val split
Generate 0 samples from train split
Generate 0 samples from val split
focal_loss alpha: [ 0.   1.   1.   1.   2.   2.5  1.   3.   1.   1.   1.   1.  10.   2.5]
loading data frame...
0it [00:00, ?it/s]
Using 0 data frame from train split
loading data frame...
0it [00:00, ?it/s]
Using 0 data frame from val split
Generate 0 samples from train split
Generate 0 samples from val split
[IOU EVAL] IGNORE:  tensor([0])
[IOU EVAL] INCLUDE:  tensor([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13])
[IOU EVAL] IGNORE:  tensor([0])
[IOU EVAL] INCLUDE:  tensor([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13])
/home/yczhou/anaconda3/envs/buct-bishe/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:136: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
===init env success===
[IOU EVAL] IGNORE:  tensor([0])
[IOU EVAL] INCLUDE:  tensor([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13])
[IOU EVAL] IGNORE:  tensor([0])
[IOU EVAL] INCLUDE:  tensor([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13])
/home/yczhou/anaconda3/envs/buct-bishe/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:136: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
[IOU EVAL] IGNORE:  ===init env success===
tensor([0])
[IOU EVAL] INCLUDE:  tensor([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13])
[IOU EVAL] IGNORE:  tensor([0])
[IOU EVAL] INCLUDE:  tensor([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13])
/home/yczhou/anaconda3/envs/buct-bishe/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:136: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
===init env success===
Traceback (most recent call last):
  File "main.py", line 148, in <module>
    exp.run()
  File "main.py", line 99, in run
    self.trainer.run(epoch, mode="Train")
  File "/home/yczhou/PMF-master/tasks/sensat_urban/pmf/trainer.py", line 528, in run
    "Acc": mean_acc.item(),
UnboundLocalError: local variable 'mean_acc' referenced before assignment
Traceback (most recent call last):
  File "main.py", line 148, in <module>
    exp.run()
  File "main.py", line 99, in run
    self.trainer.run(epoch, mode="Train")
  File "/home/yczhou/PMF-master/tasks/sensat_urban/pmf/trainer.py", line 528, in run
    "Acc": mean_acc.item(),
UnboundLocalError: local variable 'mean_acc' referenced before assignment
Traceback (most recent call last):
  File "main.py", line 148, in <module>
    exp.run()
  File "main.py", line 99, in run
    self.trainer.run(epoch, mode="Train")
  File "/home/yczhou/PMF-master/tasks/sensat_urban/pmf/trainer.py", line 448, in run
    tag="{}_lr".format(mode), scalar_value=lr, global_step=epoch)
UnboundLocalError: local variable 'lr' referenced before assignment
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 48867) of binary: /home/yczhou/anaconda3/envs/buct-bishe/bin/python
Traceback (most recent call last):
  File "/home/yczhou/anaconda3/envs/buct-bishe/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/yczhou/anaconda3/envs/buct-bishe/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/yczhou/anaconda3/envs/buct-bishe/lib/python3.7/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/home/yczhou/anaconda3/envs/buct-bishe/lib/python3.7/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/home/yczhou/anaconda3/envs/buct-bishe/lib/python3.7/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/home/yczhou/anaconda3/envs/buct-bishe/lib/python3.7/site-packages/torch/distributed/run.py", line 718, in run
    )(*cmd_args)
  File "/home/yczhou/anaconda3/envs/buct-bishe/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/yczhou/anaconda3/envs/buct-bishe/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 247, in launch_agent
    failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
main.py FAILED
------------------------------------------------------------
Failures:
[1]:
  time      : 2022-04-15_18:58:42
  host      : zkti
  rank      : 1 (local_rank: 1)
  exitcode  : 1 (pid: 48868)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[2]:
  time      : 2022-04-15_18:58:42
  host      : zkti
  rank      : 2 (local_rank: 2)
  exitcode  : 1 (pid: 48869)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2022-04-15_18:58:42
  host      : zkti
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 48867)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================

Hello author, according to the read_calib(calib_path) function of pc_processor->dataset->semantc_kitti->parser in your code,

def read_calib(calib_path):
    """
    :param calib_path: Path to a calibration text file.
    :return: dict with calibration matrices.
    """
    calib_all = {}
    with open(calib_path, 'r') as f:
        for line in f.readlines():
            if line == '\n':
                break
            key, value = line.split(':', 1)
            calib_all[key] = np.array([float(x) for x in value.split()])
calib_out = {}
# 3x4 projection matrix for left camera
calib_out['P2'] = calib_all['P2'].reshape(3, 4)
calib_out['Tr'] = np.identity(4)  # 4x4 matrix
calib_out['Tr'][:3, :4] = calib_all['Tr'].reshape(3, 4)
return calib_out

Does your calib.txt file already contain the ''Tr'' value? Why do I download only '''P0''', '''P1''', '''P2'', '''''P3' from the official website?

Thanks in advance for sharing your code for public research. The code is very standardized and easy to understand.

For my understanding, ./pc_processor/dataset/preprocess/augmentor.py is to preform PC augmenting operation such as flip, translation, etc. However, my concern is whether this will break image matching ( Lidar to camera2 matrix). Then I noticed that you have set the value of pcd_aug to false in ./tasks/pmf/trainer.py.

My questions:

1. Do you adopt pcd_aug in training?

2. In what situations should we adopt pcd_aug?

Looking forward to your reply, thanks.

Hello Dear author, I only found the lidar data of the semanti kitti dataset, can I get its corresponding images dataset download link? or Nuscenes，... Sorry, I really can't find it.

Thanks!

Hi~

Thanks for the open-source repo of your excellent work!

I notice that PMF-ResNet50 significantly outperforms PMF-ResNet34 on nuScenes Validation Set, and you even adopt ResNet101 on SensatUrban Test Set.

However, the result of PMF-ResNet50 (or deeper backbone) on SemanticKITTI Validation Set is unavailable. Did you try it before? Intuitively, it will also bring gains. Or did I miss something important?

Can someone help me with the dataset structure of Semantic Kitti and where can I find the datasets, and how many images to be used per sequence of left and right?

I am very confused

Hello author, thank you for your contribution to the community! I would like to ask you how to implement adversarial analysis of camera-only FCN based methods in ablation experiments, I am strongly curious. In my work, I want to implement it, if possible, I hope you can share the source code of the relevant part, or would you like to provide the prediction visualization results of 002777.png of the validation set 08? As shown below:

Thank you very much for your work， and I am getting errors like this during training： only one element tensors can be converted to Python scalars

It can be trained normally at the beginning, but this error is reported after a few hundred iters. Have you ever met? HELP

Hi!

One question really haunts me about the experimental results of nuScenes. How to assign semantic prediction values to points projected beyond the images?

Looking forward to your reply.

Hello, is there any code to calculate the speed of model inference in the modified project? I want to evaluate the fps indicator of the model.Thank you very much！