MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera

Felix Wimbauer

Last update: Jan 6, 2023

Related tags

Deep Learning MonoRec

Overview

MonoRec

Paper | Video (CVPR) | Video (Reconstruction) | Project Page

This repository is the official implementation of the paper:

MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera

Felix Wimbauer*, Nan Yang*, Lukas Von Stumberg, Niclas Zeller and Daniel Cremers

CVPR 2021 (arXiv)

If you find our work useful, please consider citing our paper:

@InProceedings{wimbauer2020monorec,
  title = {{MonoRec}: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera},
  author = {Wimbauer, Felix and Yang, Nan and von Stumberg, Lukas and Zeller, Niclas and Cremers, Daniel},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2021},
}

🏗️ ️ Setup

The conda environment for this project can be setup by running the following command:

conda env create -f environment.yml

🏃 Running the Example Script

We provide a sample from the KITTI Odometry test set and a script to run MonoRec on it in example/. To download the pretrained model and put it into the right place, run download_model.sh. You can manually do this by can by downloading the weights from here and unpacking the file to saved/checkpoints/monorec_depth_ref.pth. The example script will plot the keyframe, depth prediction and mask prediction.

cd example
python test_monorec.py

🗃️ Data

In all of our experiments we used the KITTI Odometry dataset for training. For additional evaluations, we used the KITTI, Oxford RobotCar, TUM Mono-VO and TUM RGB-D datasets. All datapaths can be specified in the respective configuration files. In our experiments, we put all datasets into a seperate folder ../data.

KITTI Odometry

To setup KITTI Odometry, download the color images and calibration files from the official website (around 145 GB). Instead of the given velodyne laser data files, we use the improved ground truth depth for evaluation, which can be downloaded from here.

Unzip the color images and calibration files into ../data. The lidar depth maps can be extracted into the given folder structure by running data_loader/scripts/preprocess_kitti_extract_annotated_depth.py.

For training and evaluation, we use the poses estimated by Deep Virtual Stereo Odometry (DVSO). They can be downloaded from here and should be placed under ../data/{kitti_path}/poses_dso. This folder structure is ensured when unpacking the zip file in the {kitti_path} directory.

The auxiliary moving object masks can be downloaded from here. They should be placed under ../data/{kitti_path}/sequences/{seq_num}/mvobj_mask. This folder structure is ensured when unpacking the zip file in the {kitti_path} directory.

Oxford RobotCar

To setup Oxford RobotCar, download the camera model files and the large sample from the official website. Code, as well as, camera extrinsics need to be downloaded from the official GitHub repository. Please move the content of the python folder to data_loaders/oxford_robotcar/. extrinsics/, models/ and sample/ need to be moved to ../data/oxford_robotcar/. Note that for poses we use the official visual odometry poses, which are not provided in the large sample. They need to be downloaded manually from the raw dataset and unpacked into the sample folder.

TUM Mono-VO

Unfortunately, TUM Mono-VO images are provided only in the original, distorted form. Therefore, they need to be undistorted first before fed into MonoRec. To obtain poses for the sequences, we run the publicly available version of Direct Sparse Odometry.

TUM RGB-D

The official sequences can be downloaded from the official website and need to be unpacked under ../data/tumrgbd/{sequence_name}. Note that our provided dataset implementation assumes intrinsics from fr3 sequences. Note that the data loader for this dataset also relies on the code from the Oxford Robotcar dataset.

🏋️ Training & Evaluation

Please stay tuned! Training code will be published soon!

We provide checkpoints for each training stage:

Training stage	Download
Depth Bootstrap	Link
Mask Bootstrap	Link
Mask Refinement	Link
Depth Refinement (final model)	Link

Run download_model.sh to download the final model. It will automatically get moved to saved/checkpoints.

To reproduce the evaluation results on different datasets, run the following commands:

python evaluate.py --config configs/evaluate/eval_monorec.json        # KITTI Odometry
python evaluate.py --config configs/evaluate/eval_monorec_oxrc.json   # Oxford Robotcar

☁️ Pointclouds

To reproduce the pointclouds depicted in the paper and video, use the following commands:

python create_pointcloud.py --config configs/test/pointcloud_monorec.json       # KITTI Odometry
python create_pointcloud.py --config configs/test/pointcloud_monorec_oxrc.json  # Oxford Robotcar
python create_pointcloud.py --config configs/test/pointcloud_monorec_tmvo.json  # TUM Mono-VO

Comments

TypeError: 'tuple' object is not callable

when train the model with command python train.py --config configs/train/monorec/monorec_depth.json --options stereo, get an error:

Traceback (most recent call last):
  File "/home/hyel/python_work/MonoRec/train.py", line 70, in <module>
    main(config, config.args.options)
  File "/home/hyel/python_work/MonoRec/train.py", line 49, in main
    trainer.train()
  File "/home/hyel/python_work/MonoRec/base/base_trainer.py", line 73, in train
    result = self._train_epoch(epoch)
  File "/home/hyel/python_work/MonoRec/trainer/trainer.py", line 84, in _train_epoch
    for batch_idx, (data, target) in enumerate(self.data_loader):
  File "/home/hyel/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 517, in __next__
    data = self._next_data()
  File "/home/hyel/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1199, in _next_data
    return self._process_data(data)
  File "/home/hyel/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1225, in _process_data
    data.reraise()
  File "/home/hyel/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/_utils.py", line 429, in reraise
    raise self.exc_type(msg)
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/hyel/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 202, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/hyel/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/hyel/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/hyel/python_work/MonoRec/data_loader/kitti_odometry_dataset.py", line 250, in __getitem__
    self._crop_boxes[dataset_index])
  File "/home/hyel/python_work/MonoRec/data_loader/kitti_odometry_dataset.py", line 126, in preprocess_image
    img = self.color_transform(img)
  File "/home/hyel/python_work/MonoRec/data_loader/kitti_odometry_dataset.py", line 383, in __call__
    return map_fn(x, self.transform)
  File "/home/hyel/python_work/MonoRec/utils/util.py", line 22, in map_fn
    return fn(batch)
TypeError: 'tuple' object is not callable

modify function in kitti_odometry_dataset.py from:

class ColorJitterMulti(torchvision.transforms.ColorJitter):
    def fix_transform(self):
        self.transform = self.get_params(self.brightness, self.contrast,
                                         self.saturation, self.hue)

    def __call__(self, x):
        return map_fn(x, self.transform)

class ColorJitterMulti(torchvision.transforms.ColorJitter):
    def fix_transform(self):
        self.transform = self.get_params(self.brightness, self.contrast,
                                         self.saturation, self.hue)

    def __call__(self, x):
        return map_fn(x, self.forward)           # change self.transform to self.forward

can fix this error

this is a bug? is it right to modify the code like this?

opened by huangbinz 6

Different evaluation results are obtained on different GPUs

Hi, Thanks for your amazing work!

I recently encountered a problem, I ran the evaluation of the KITTI dataset on different GPUs and they got different results from the run.

The GPU I used are：

Tesla P100-16GB (provided by Colab)
RTX 3090-24GB (desktop)
RTX 3060-8GB (laptop)

The version of dependency packages are:

torch                     1.11.0+cu113             pypi_0    pypi
torchaudio                0.11.0+cu113             pypi_0    pypi
torchvision               0.12.0+cu113             pypi_0    pypi
opencv-python             4.1.2.30                 pypi_0    pypi
python                    3.7.13               h12debd9_0
scipy                     1.4.1                    pypi_0    pypi
scikit-image              0.18.3                   pypi_0    pypi
pykitti                   0.3.1                    pypi_0    pypi
kornia                    0.6.4                    pypi_0    pypi

The final metric results are as follows:

## RTX 3090
'loss': 0.0, 'metrics': [0.15538795390363738, 2.7975851661019338, 5.625108310379362, 0.18630796428342675, 0.8499496491823949, 0.9419053045479213, 0.9715246053337913], 'metrics_correct': [0.1553879539036371, 2.797585166101927, 5.625108310379343, 0.18630796428342658, 0.849949649182393, 0.941905304547917, 0.9715246053337869], 'valid_batches': 4317.0, 'loss_loss': 0.0, 'metrics_info': ['abs_rel_sparse_metric', 'sq_rel_sparse_metric', 'rmse_sparse_metric', 'rmse_log_sparse_metric', 'a1_sparse_metric', 'a2_sparse_metric', 'a3_sparse_metric']

## RTX3060
loss': 0.0, 'metrics': [0.15538795390363738, 2.7975851661019338, 5.625108310379362, 0.18630796428342675, 0.8499496491823949, 0.9419053045479213, 0.9715246053337913], 'metrics_correct': [0.1553879539036371, 2.797585166101927, 5.625108310379343, 0.18630796428342658, 0.849949649182393, 0.941905304547917, 0.9715246053337869], 'valid_batches': 4317.0, 'loss_loss': 0.0, 'metrics_info': ['abs_rel_sparse_metric', 'sq_rel_sparse_metric', 'rmse_sparse_metric', 'rmse_log_sparse_metric', 'a1_sparse_metric', 'a2_sparse_metric', 'a3_sparse_metric']

## P100 (provided by colab)
'loss': 0.0, 'metrics': [0.050102306426327375, 0.29000417841479736, 2.2656219501116497, 0.08224765892285756, 0.9723125736912782, 0.9907187367026828, 0.9957081838727743], 'metrics_correct': [0.0501023064263273, 0.29000417841479575, 2.2656219501116426, 0.08224765892285742, 0.9723125736912774, 0.9907187367026803, 0.9957081838727698], 'valid_batches': 4317.0, 'loss_loss': 0.0, 'metrics_info': ['abs_rel_sparse_metric', 'sq_rel_sparse_metric', 'rmse_sparse_metric', 'rmse_log_sparse_metric', 'a1_sparse_metric', 'a2_sparse_metric', 'a3_sparse_metric']

Full evaluation logs are attached as bellow: colab_p100_eval_log.txt RTX3090_eval_log.txt RTX3060_eval_log.txt

I found that 3090 and 3060 run with the same result, but it differs greatly from the result of P100, which is very closed to the result provided by the paper.

I did not change any code during the above evaluation process, and they use the same data set.

Have you encountered this problem? Could you please give me some advice?

opened by neil0306 5

Tool for reconstructed scene visualization

Hi!

Thanks for this amazing job :) I am wondering if you could kindly inform which tool you are using for the reconstructed scene visualization, it looks really cool ! Thanks!

opened by HanzhiC 5

example script list index error

I was trying to run the example script, but ran into an error:

Traceback (most recent call last):
File "test_monorec.py", line 41, in <module>
batch, depth = dataset.__getitem__(index)
File "../data_loader/kitti_odometry_dataset.py", line 252, in __getitem__
keyframe_pose = torch.tensor(dataset.poses[index + self._offset], dtype=torch.float32)
IndexError: list index out of range

I found out that dataset poses for sequence 07 is an empty list. In this code below in kitti_odometry_dataset.py string constant is "poses_dvso". However, examples poses dir has name "poses_dso".

if self.use_dso_poses:
    for dataset in self._datasets:
          dataset.pose_path = self.dataset_dir / "poses_dvso"
          dataset._load_poses()

It seems that changing the constant to "poses_dso" fixes the error.

opened by caisilus 4

DVSO keypoints as metric depth?

Hi, thanks for sharing this amazing work.

I have a question on the DVSO keypoints you provide in the readme. As far as I understood, they actually contain disparity values, which you convert to depth as follows:

https://github.com/Brummi/MonoRec/blob/671fec524fec0b21720d098f14ee18faab9d291d/data_loader/kitti_odometry_dataset.py#L162

The conversion depth = (baseline * focal) / disparity is fine for me, but I don't really understand how the width and the 65535 interact in this formula. Could you clarify a bit better how to obtain metric depth values from these keypoints?

Thank you in advance.

opened by morsingher 4
train "monorec_depth_ref" problem
Question 1:

When I run the last training python train_monorec.py --config configs/train/monorec/monorec_depth_ref.json --options stereo stereo_repr it can't be continued. Train Epoch: 1 [12800/13714 (93%)] Loss: 0.340444 Loss_dict: {'sdl_0': tensor(0.0036, device='cuda:0'), 'static_md2l_0': tensor(0.0692, device='cuda:0'), 'dynamic_md2l_0': tensor(0.1163, device='cuda:0'), 'md2l_0': tensor(0.0693, device='cuda:0'), 'sdl_1': tensor(0.0036, device='cuda:0'), 'static_md2l_1': tensor(0.0695, device='cuda:0'), 'dynamic_md2l_1': tensor(0.1168, device='cuda:0'), 'md2l_1': tensor(0.0696, device='cuda:0'), 'sdl_2': tensor(0.0036, device='cuda:0'), 'static_md2l_2': tensor(0.0702, device='cuda:0'), 'dynamic_md2l_2': tensor(0.1136, device='cuda:0'), 'md2l_2': tensor(0.0703, device='cuda:0'), 'sdl_3': tensor(0.0039, device='cuda:0'), 'static_md2l_3': tensor(0.0722, device='cuda:0'), 'dynamic_md2l_3': tensor(0.1166, device='cuda:0'), 'md2l_3': tensor(0.0722, device='cuda:0'), 'loss': tensor(0.3404, device='cuda:0')} hold on here

Question 2:

Could you provide your GPU configuration and training time?
opened by zhowying 4
FileNotFoundError

Run "python train_monorec.py --config configs/train/monorec/monorec_mask.json --options stereo"

File "/home/user/monorec/MonoRec-main-test/data_loader/kitti_odometry_dataset.py", line 67, in init with open(self.dataset_dir / "sequences" / sequence / (index_mask_name + ".json")) as f: FileNotFoundError: [Errno 2] No such file or directory: '../data/dataset/sequences/01/index_mask.json'

opened by zhowying 4
How to run on my own data

Hi.First of all, thank you for your efforts, and I am also surprised by the effect. If I want to make predictions in my own data, how should I get the configs

opened by JustC0deIt 3
Why set temporal frames from default 2 to 10 when 'lidar_depth' and 'annotated_lidar' are enabled?

Hi,

Thank you al lot for your impressive work.

I found a place that I do not understand. As shown in the code below https://github.com/Brummi/MonoRec/blob/b2b3decc130d9ca333d1350096b9838a12f977a3/data_loader/kitti_odometry_dataset.py#L56-L58

why set the nearby frames from default 2 to at least 10 when 'lidar_depth' and 'annotated_lidar' are enabled. Is it a general setting for both training and testing or only a setting for testing? Thank you!

opened by ruili3 3
attribute lookup CalibData on pykitti.odometry failed

Hi,when I set up the dataset,runing the order python train.py --config configs/train/monorec/monorec_depth.json --options stereo
"enumerate(self.data_loader)" will alert pickle.PicklingError: Can't pickle <class 'pykitti.odometry.CalibData'>: attribute lookup CalibData on pykitti.odometry failed

opened by Tangccwu 3

TUM RGBD Point Cloud Issues

I tried to run the TUM RGBD Dataset using the following config file

{
  "name": "Pointcloud Creation",
  "n_gpu": 8,
  "output_dir": "saved/pointclouds/monorec",
  "file_name": "tmrgbd.ply",
  "roi": [
    160,
    480,
    40,
    600
  ],
  "start": 0,
  "end": 300,
  "max_d": 20,
  "use_mask": true,
  "arch": {
    "type": "MonoRecModel",
    "args": {
      "pretrain_mode": 0,
      "checkpoint_location": [
          "saved/checkpoints/monorec_depth_ref.pth"
      ]
    }
  },
  "data_set": {
    "type": "TUMRGBDDataset",
    "args": {
      "dataset_dir": "data/tumrgbd/",
      "frame_count": 2,
      "target_image_size": [
        480,
        640
      ],
      "dilation": 1
    }
  }
}

Dataset: https://vision.in.tum.de/rgbd/dataset/freiburg3/rgbd_dataset_freiburg3_long_office_household.tgz

I'm using FR3 dataset since the intrinsic parameters were set already for this in the module

https://github.com/Brummi/MonoRec/blob/1d05d2483a270fb3b9a7f06e8dc8259760c0305d/data_loader/tum_rgbd_dataset.py#L25-L30

But these are the results i get from the point cloud

Screenshot from 2021-07-13 17-47-48

This was the output on console


<THE MODEL ARCH OUTPUT>

        (1): ConvReLU2(
          (pad_0): PadSameConv2d()
          (conv_y): Conv2d(64, 64, kernel_size=(3, 1), stride=(1, 1))
          (leaky_relu): LeakyReLU(negative_slope=0.1)
          (pad_1): PadSameConv2d()
          (conv_x): Conv2d(64, 64, kernel_size=(1, 3), stride=(1, 1))
        )
      )
      (3): Refine(
        (conv2d_t): ConvTranspose2d(192, 48, kernel_size=(4, 4), stride=(2, 2))
        (pad): PadSameConv2dTransposed()
        (leaky_relu): LeakyReLU(negative_slope=0.1)
      )
      (4): Sequential(
        (0): ConvReLU2(
          (pad_0): PadSameConv2d()
          (conv_y): Conv2d(96, 32, kernel_size=(3, 1), stride=(1, 1))
          (leaky_relu): LeakyReLU(negative_slope=0.1)
          (pad_1): PadSameConv2d()
          (conv_x): Conv2d(32, 32, kernel_size=(1, 3), stride=(1, 1))
        )
        (1): PadSameConv2d()
        (2): Conv2d(32, 24, kernel_size=(3, 3), stride=(1, 1))
        (3): LeakyReLU(negative_slope=0.1)
      )
    )
    (predictors): ModuleList(
      (0): Sequential(
        (0): PadSameConv2d()
        (1): Conv2d(256, 1, kernel_size=(3, 3), stride=(1, 1))
      )
      (1): Sequential(
        (0): PadSameConv2d()
        (1): Conv2d(128, 1, kernel_size=(3, 3), stride=(1, 1))
      )
      (2): Sequential(
        (0): PadSameConv2d()
        (1): Conv2d(64, 1, kernel_size=(3, 3), stride=(1, 1))
      )
      (3): Sequential(
        (0): PadSameConv2d()
        (1): Conv2d(24, 1, kernel_size=(3, 3), stride=(1, 1))
      )
    )
  )
)
  0%|                                                                   | 0/300 [00:00<?, ?it/s]/opt/conda/conda-bld/pytorch_1587428398394/work/torch/csrc/utils/tensor_numpy.cpp:141: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program.
/opt/conda/conda-bld/pytorch_1587428398394/work/torch/csrc/utils/tensor_numpy.cpp:141: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program.
/opt/conda/conda-bld/pytorch_1587428398394/work/torch/csrc/utils/tensor_numpy.cpp:141: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program.
/opt/conda/conda-bld/pytorch_1587428398394/work/torch/csrc/utils/tensor_numpy.cpp:141: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program.
/opt/conda/conda-bld/pytorch_1587428398394/work/torch/csrc/utils/tensor_numpy.cpp:141: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program.
/opt/conda/conda-bld/pytorch_1587428398394/work/torch/csrc/utils/tensor_numpy.cpp:141: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program.
/opt/conda/conda-bld/pytorch_1587428398394/work/torch/csrc/utils/tensor_numpy.cpp:141: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program.
/opt/conda/conda-bld/pytorch_1587428398394/work/torch/csrc/utils/tensor_numpy.cpp:141: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program.
/opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:3226: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
  warnings.warn("Default grid_sample and affine_grid behavior has changed "
/opt/conda/lib/python3.7/site-packages/torch/nn/functional.py:1558: UserWarning: nn.functional.tanh is deprecated. Use torch.tanh instead.
  warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")
100%|█████████████████████████████████████████████████████████| 300/300 [02:03<00:00,  2.42it/s]

These are my dependencies

torchvision==0.6.0
jupyterlab
tensorboard
tensorboardX
opencv-python==4.2.0.34
scipy==1.7.0
kornia==0.3.2
scikit-image==0.18.2
tqdm==4.61.2
pykitti==0.3.1
colour-demosaicing==0.1.6

and i'm using Pytorch 1.5.0 from docker pytorch/pytorch:1.5-cuda10.1-cudnn7-devel

opened by satyajitghana 3

Mask module bootstrap does not converge

Hi:

Thank you for sharing this amazing work!

I tried to train the model several times but the mask module bootstrapping can not converge on the KITTI dataset. The only thing I changed in the code is from here.

The training iou is increasing but the validation iou is decreasing so is it overfitting?

The result looks very noisy even in the training set:

However, the mask module refinement converged as well as claimed in the paper.

below is the training log from mask module bootstrapping:

2022-05-21 18:56:28,980 - train 2022-05-21 19:33:24,088 - trainer - INFO - 2022-05-21 19:33:24,089 - trainer - INFO - 2022-05-21 19:33:24,090 - trainer - INFO - 2022-05-21 19:33:24,091 - trainer - INFO - 2022-05-21 19:33:24,092 - trainer - INFO - 2022-05-21 19:33:24,092 - trainer - INFO - 2022-05-21 19:33:24,093 - trainer - INFO - 2022-05-21 19:33:24,094 - trainer - INFO - 2022-05-21 19:33:24,095 - trainer - INFO - 2022-05-21 19:33:24,096 - trainer - INFO - 2022-05-21 19:33:24,096 - trainer - INFO - 2022-05-21 19:33:24,097 - trainer - INFO - 2022-05-21 19:33:24,098 - trainer - INFO - 2022-05-21 19:33:24,099 - trainer - INFO - 2022-05-21 19:33:24,100 - trainer - INFO - 2022-05-21 19:33:24,100 - trainer - INFO - 2022-05-21 19:33:24,101 - trainer - INFO - 2022-05-21 19:33:24,102 - trainer - INFO - 2022-05-21 19:33:24,103 - trainer - INFO - 2022-05-21 19:33:24,324 - trainer 2022-05-21 19:33:24,545 - trainer 2022-05-21 20:09:51,592 - trainer - INFO - 2022-05-21 20:09:51,593 - trainer - INFO - 2022-05-21 20:09:51,596 - trainer - INFO - 2022-05-21 20:09:51,599 - trainer - INFO - 2022-05-21 20:09:51,601 - trainer - INFO - 2022-05-21 20:09:51,604 - trainer - INFO - 2022-05-21 20:09:51,606 - trainer - INFO - 2022-05-21 20:09:51,609 - trainer - INFO - 2022-05-21 20:09:51,612 - trainer - INFO - 2022-05-21 20:09:51,614 - trainer - INFO - 2022-05-21 20:09:51,617 - trainer - INFO - 2022-05-21 20:09:51,619 - trainer - INFO - 2022-05-21 20:09:51,621 - trainer - INFO - 2022-05-21 20:09:51,623 - trainer - INFO - 2022-05-21 20:09:51,625 - trainer - INFO - 2022-05-21 20:09:51,627 - trainer - INFO - 2022-05-21 20:09:51,629 - trainer - INFO - 2022-05-21 20:09:51,631 - trainer - INFO - 2022-05-21 20:09:51,633 - trainer - INFO - 2022-05-21 20:09:52,095 - trainer 2022-05-21 20:46:30,289 - trainer - INFO - 2022-05-21 20:46:30,292 - trainer - INFO - 2022-05-21 20:46:30,298 - trainer - INFO - 2022-05-21 20:46:30,303 - trainer - INFO - 2022-05-21 20:46:30,308 - trainer - INFO - 2022-05-21 20:46:30,313 - trainer - INFO - 2022-05-21 20:46:30,318 - trainer - INFO - 2022-05-21 20:46:30,323 - trainer - INFO - 2022-05-21 20:46:30,329 - trainer - INFO - 2022-05-21 20:46:30,332 - trainer - INFO - 2022-05-21 20:46:30,337 - trainer - INFO - 2022-05-21 20:46:30,342 - trainer - INFO - 2022-05-21 20:46:30,347 - trainer - INFO - 2022-05-21 20:46:30,353 - trainer - INFO - 2022-05-21 20:46:30,358 - trainer - INFO - 2022-05-21 20:46:30,364 - trainer - INFO - 2022-05-21 20:46:30,371 - trainer - INFO - 2022-05-21 20:46:30,377 - trainer - INFO - 2022-05-21 20:46:30,382 - trainer - INFO - 2022-05-21 20:46:30,914 - trainer 2022-05-21 21:22:54,544 - trainer - INFO - 2022-05-21 21:22:54,551 - trainer - INFO - 2022-05-21 21:22:54,571 - trainer - INFO - 2022-05-21 21:22:54,589 - trainer - INFO - 2022-05-21 21:22:54,608 - trainer - INFO - 2022-05-21 21:22:54,621 - trainer - INFO - 2022-05-21 21:22:54,638 - trainer - INFO - 2022-05-21 21:22:54,657 - trainer - INFO - 2022-05-21 21:22:54,675 - trainer - INFO - 2022-05-21 21:22:54,692 - trainer - INFO - 2022-05-21 21:22:54,709 - trainer - INFO - 2022-05-21 21:22:54,726 - trainer - INFO - 2022-05-21 21:22:54,741 - trainer - INFO - 2022-05-21 21:22:54,756 - trainer - INFO - 2022-05-21 21:22:54,772 - trainer - INFO - 2022-05-21 21:22:54,786 - trainer - INFO - 2022-05-21 21:22:54,802 - trainer - INFO - 2022-05-21 21:22:54,818 - trainer - INFO - 2022-05-21 21:22:54,834 - trainer - INFO - 2022-05-21 21:22:55,087 - trainer 2022-05-21 21:59:20,427 - trainer - INFO - 2022-05-21 21:59:20,429 - trainer - INFO - 2022-05-21 21:59:20,430 - trainer - INFO - 2022-05-21 21:59:20,431 - trainer - INFO - 2022-05-21 21:59:20,432 - trainer - INFO - 2022-05-21 21:59:20,432 - trainer - INFO - 2022-05-21 21:59:20,433 - trainer - INFO - 2022-05-21 21:59:20,434 - trainer - INFO - 2022-05-21 21:59:20,435 - trainer - INFO - 2022-05-21 21:59:20,436 - trainer - INFO - 2022-05-21 21:59:20,437 - trainer - INFO - 2022-05-21 21:59:20,437 - trainer - INFO - 2022-05-21 21:59:20,438 - trainer - INFO - 2022-05-21 21:59:20,439 - trainer - INFO - 2022-05-21 21:59:20,440 - trainer - INFO - 2022-05-21 21:59:20,441 - trainer - INFO - 2022-05-21 21:59:20,441 - trainer - INFO - 2022-05-21 21:59:20,442 - trainer - INFO - 2022-05-21 21:59:20,443 - trainer - INFO - 2022-05-21 21:59:20,643 - trainer 2022-05-21 22:35:45,561 - trainer - INFO - 2022-05-21 22:35:45,562 - trainer - INFO - 2022-05-21 22:35:45,563 - trainer - INFO - 2022-05-21 22:35:45,564 - trainer - INFO - 2022-05-21 22:35:45,565 - trainer - INFO - 2022-05-21 22:35:45,566 - trainer - INFO - 2022-05-21 22:35:45,567 - trainer - INFO - 2022-05-21 22:35:45,567 - trainer - INFO - 2022-05-21 22:35:45,568 - trainer - INFO - 2022-05-21 22:35:45,569 - trainer - INFO - 2022-05-21 22:35:45,570 - trainer - INFO - 2022-05-21 22:35:45,571 - trainer - INFO - 2022-05-21 22:35:45,572 - trainer - INFO - 2022-05-21 22:35:45,573 - trainer - INFO - 2022-05-21 22:35:45,573 - trainer - INFO - 2022-05-21 22:35:45,574 - trainer - INFO - 2022-05-21 22:35:45,575 - trainer - INFO - 2022-05-21 22:35:45,576 - trainer - INFO - 2022-05-21 22:35:45,577 - trainer - INFO - 2022-05-21 22:35:45,747 - trainer 2022-05-21 23:12:12,250 - trainer - INFO - 2022-05-21 23:12:12,253 - trainer - INFO - 2022-05-21 23:12:12,259 - trainer - INFO - 2022-05-21 23:12:12,266 - trainer - INFO - 2022-05-21 23:12:12,272 - trainer - INFO - 2022-05-21 23:12:12,278 - trainer - INFO - 2022-05-21 23:12:12,283 - trainer - INFO - 2022-05-21 23:12:12,288 - trainer - INFO - 2022-05-21 23:12:12,293 - trainer - INFO - 2022-05-21 23:12:12,297 - trainer - INFO - 2022-05-21 23:12:12,300 - trainer - INFO - 2022-05-21 23:12:12,304 - trainer - INFO - 2022-05-21 23:12:12,308 - trainer - INFO - 2022-05-21 23:12:12,312 - trainer - INFO - 2022-05-21 23:12:12,316 - trainer - INFO - 2022-05-21 23:12:12,320 - trainer - INFO - 2022-05-21 23:12:12,325 - trainer - INFO - 2022-05-21 23:12:12,328 - trainer - INFO - 2022-05-21 23:12:12,331 - trainer - INFO - 2022-05-21 23:12:12,700 - trainer 2022-05-21 23:48:32,490 - trainer - INFO - 2022-05-21 23:48:32,503 - trainer - INFO - 2022-05-21 23:48:32,553 - trainer - INFO - 2022-05-21 23:48:32,580 - trainer - INFO - 2022-05-21 23:48:32,610 - trainer - INFO - 2022-05-21 23:48:32,640 - trainer - INFO - 2022-05-21 23:48:32,670 - trainer - INFO - 2022-05-21 23:48:32,701 - trainer - INFO - 2022-05-21 23:48:32,733 - trainer - INFO - 2022-05-21 23:48:32,764 - trainer - INFO - 2022-05-21 23:48:32,795 - trainer - INFO - 2022-05-21 23:48:32,825 - trainer - INFO - 2022-05-21 23:48:32,855 - trainer - INFO - 2022-05-21 23:48:32,866 - trainer - INFO - 2022-05-21 23:48:32,870 - trainer - INFO - 2022-05-21 23:48:32,885 - trainer - INFO - 2022-05-21 23:48:32,913 - trainer - INFO - 2022-05-21 23:48:32,939 - trainer - INFO - 2022-05-21 23:48:32,964 - trainer - INFO - 2022-05-21 23:48:33,342 - trainer 2022-05-22 00:24:59,842 - trainer - INFO - 2022-05-22 00:24:59,843 - trainer - INFO - 2022-05-22 00:24:59,844 - trainer - INFO - 2022-05-22 00:24:59,845 - trainer - INFO - 2022-05-22 00:24:59,846 - trainer - INFO - 2022-05-22 00:24:59,847 - trainer - INFO - 2022-05-22 00:24:59,847 - trainer - INFO - 2022-05-22 00:24:59,848 - trainer - INFO - 2022-05-22 00:24:59,849 - trainer - INFO - 2022-05-22 00:24:59,850 - trainer - INFO - 2022-05-22 00:24:59,851 - trainer - INFO - 2022-05-22 00:24:59,851 - trainer - INFO - 2022-05-22 00:24:59,852 - trainer - INFO - 2022-05-22 00:24:59,853 - trainer - INFO - 2022-05-22 00:24:59,854 - trainer - INFO - 2022-05-22 00:24:59,855 - trainer - INFO - 2022-05-22 00:24:59,856 - trainer - INFO - 2022-05-22 00:24:59,856 - trainer - INFO - 2022-05-22 00:24:59,857 - trainer - INFO - 2022-05-22 00:25:00,027 - trainer 2022-05-22 01:01:40,023 - trainer - INFO - 2022-05-22 01:01:40,027 - trainer - INFO - 2022-05-22 01:01:40,037 - trainer - INFO - 2022-05-22 01:01:40,048 - trainer - INFO - 2022-05-22 01:01:40,059 - trainer - INFO - 2022-05-22 01:01:40,069 - trainer - INFO - 2022-05-22 01:01:40,078 - trainer - INFO - 2022-05-22 01:01:40,087 - trainer - INFO - 2022-05-22 01:01:40,096 - trainer - INFO - 2022-05-22 01:01:40,105 - trainer - INFO - 2022-05-22 01:01:40,113 - trainer - INFO - 2022-05-22 01:01:40,122 - trainer - INFO - 2022-05-22 01:01:40,131 - trainer - INFO - 2022-05-22 01:01:40,141 - trainer - INFO - 2022-05-22 01:01:40,150 - trainer - INFO - 2022-05-22 01:01:40,159 - trainer - INFO - 2022-05-22 01:01:40,168 - trainer - INFO - 2022-05-22 01:01:40,176 - trainer - INFO - 2022-05-22 01:01:40,184 - trainer - INFO - 2022-05-22 01:01:40,399 - trainer 2022-05-22 01:38:00,857 - trainer - INFO - 2022-05-22 01:38:00,858 - trainer - INFO - 2022-05-22 01:38:00,859 - trainer - INFO - 2022-05-22 01:38:00,859 - trainer - INFO - 2022-05-22 01:38:00,860 - trainer - INFO - 2022-05-22 01:38:00,861 - trainer - INFO - 2022-05-22 01:38:00,862 - trainer - INFO - 2022-05-22 01:38:00,863 - trainer - INFO - 2022-05-22 01:38:00,864 - trainer - INFO - 2022-05-22 01:38:00,865 - trainer - INFO - 2022-05-22 01:38:00,866 - trainer - INFO - 2022-05-22 01:38:00,867 - trainer - INFO - 2022-05-22 01:38:00,868 - trainer - INFO - 2022-05-22 01:38:00,869 - trainer - INFO - 2022-05-22 01:38:00,870 - trainer - INFO - 2022-05-22 01:38:00,871 - trainer - INFO - 2022-05-22 01:38:00,871 - trainer - INFO - 2022-05-22 01:38:00,872 - trainer - INFO - 2022-05-22 01:38:00,873 - trainer - INFO - 2022-05-22 01:38:01,035 - trainer 2022-05-22 02:14:19,960 - trainer - INFO - 2022-05-22 02:14:19,968 - trainer - INFO - 2022-05-22 02:14:19,987 - trainer - INFO - 2022-05-22 02:14:19,999 - trainer - INFO - 2022-05-22 02:14:20,012 - trainer - INFO - 2022-05-22 02:14:20,025 - trainer - INFO - 2022-05-22 02:14:20,042 - trainer - INFO - 2022-05-22 02:14:20,057 - trainer - INFO - 2022-05-22 02:14:20,073 - trainer - INFO - 2022-05-22 02:14:20,087 - trainer - INFO - 2022-05-22 02:14:20,101 - trainer - INFO - 2022-05-22 02:14:20,116 - trainer - INFO - 2022-05-22 02:14:20,129 - trainer - INFO - 2022-05-22 02:14:20,228 - trainer - INFO - 2022-05-22 02:14:20,245 - trainer - INFO - 2022-05-22 02:14:20,266 - trainer - INFO - 2022-05-22 02:14:20,287 - trainer - INFO - 2022-05-22 02:14:20,308 - trainer - INFO - 2022-05-22 02:14:20,331 - trainer - INFO - 2022-05-22 02:14:20,523 - trainer 2022-05-22 02:50:43,264 - trainer - INFO - 2022-05-22 02:50:43,266 - trainer - INFO - 2022-05-22 02:50:43,267 - trainer - INFO - 2022-05-22 02:50:43,269 - trainer - INFO - 2022-05-22 02:50:43,270 - trainer - INFO - 2022-05-22 02:50:43,272 - trainer - INFO - 2022-05-22 02:50:43,273 - trainer - INFO - 2022-05-22 02:50:43,273 - trainer - INFO - 2022-05-22 02:50:43,274 - trainer - INFO - 2022-05-22 02:50:43,275 - trainer - INFO - 2022-05-22 02:50:43,276 - trainer - INFO - 2022-05-22 02:50:43,277 - trainer - INFO - 2022-05-22 02:50:43,278 - trainer - INFO - 2022-05-22 02:50:43,279 - trainer - INFO - 2022-05-22 02:50:43,280 - trainer - INFO - 2022-05-22 02:50:43,281 - trainer - INFO - 2022-05-22 02:50:43,282 - trainer - INFO - 2022-05-22 02:50:43,283 - trainer - INFO - 2022-05-22 02:50:43,284 - trainer - INFO - 2022-05-22 02:50:43,848 - trainer 2022-05-22 03:27:08,776 - trainer - INFO - 2022-05-22 03:27:08,777 - trainer - INFO - 2022-05-22 03:27:08,779 - trainer - INFO - 2022-05-22 03:27:08,780 - trainer - INFO - 2022-05-22 03:27:08,782 - trainer - INFO - 2022-05-22 03:27:08,784 - trainer - INFO - 2022-05-22 03:27:08,785 - trainer - INFO - 2022-05-22 03:27:08,786 - trainer - INFO - 2022-05-22 03:27:08,788 - trainer - INFO - 2022-05-22 03:27:08,789 - trainer - INFO - 2022-05-22 03:27:08,790 - trainer - INFO - 2022-05-22 03:27:08,791 - trainer - INFO - 2022-05-22 03:27:08,792 - trainer - INFO - 2022-05-22 03:27:08,793 - trainer - INFO - 2022-05-22 03:27:08,794 - trainer - INFO - 2022-05-22 03:27:08,795 - trainer - INFO - 2022-05-22 03:27:08,796 - trainer - INFO - 2022-05-22 03:27:08,797 - trainer - INFO - 2022-05-22 03:27:08,798 - trainer - INFO - 2022-05-22 03:27:09,118 - trainer 2022-05-22 04:03:31,666 - trainer - INFO - 2022-05-22 04:03:31,672 - trainer - INFO - 2022-05-22 04:03:31,679 - trainer - INFO - 2022-05-22 04:03:31,687 - trainer - INFO - 2022-05-22 04:03:31,695 - trainer - INFO - 2022-05-22 04:03:31,703 - trainer - INFO - 2022-05-22 04:03:31,711 - trainer - INFO - 2022-05-22 04:03:31,720 - trainer - INFO - 2022-05-22 04:03:31,729 - trainer - INFO - 2022-05-22 04:03:31,739 - trainer - INFO - 2022-05-22 04:03:31,749 - trainer - INFO - 2022-05-22 04:03:31,754 - trainer - INFO - 2022-05-22 04:03:31,765 - trainer - INFO - 2022-05-22 04:03:31,774 - trainer - INFO - 2022-05-22 04:03:31,784 - trainer - INFO - 2022-05-22 04:03:31,795 - trainer - INFO - 2022-05-22 04:03:31,805 - trainer - INFO - 2022-05-22 04:03:31,815 - trainer - INFO - 2022-05-22 04:03:31,826 - trainer - INFO - 2022-05-22 04:03:32,004 - trainer - INFO - 1503377 trainable parameters epoch : 1 loss : 0.33975716474378714 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 1.975 rmse_sparse_metric: 1.975 loss_iou : 0.08929756164550781 loss_prec : 0.09143932342529297 loss_loss : 0.33975664774576825 loss_acc : 0.9185029093424479 loss_rec : 0.9724029541015625 val_loss : 2.165919481880135 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.09720667865541247 val_loss_prec : 0.10425782865948147 val_loss_loss : 2.165919621785482 val_loss_acc : 0.8980947070651584 val_loss_rec : 0.7917230923970541 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... - INFO - Saving current best: model_best.pth ... epoch : 2 loss : 0.18604853542870842 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 1.9338541666666667 rmse_sparse_metric: 1.9338541666666667 loss_iou : 0.13620360056559244 loss_prec : 0.13805244445800782 loss_loss : 0.1860488001505534 loss_acc : 0.9541281127929687 loss_rec : 0.9810747273763021 val_loss : 3.564138030219409 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.10033636622958714 val_loss_prec : 0.1108185847600301 val_loss_loss : 3.5641377766927085 val_loss_acc : 0.9085981580946181 val_loss_rec : 0.7776173485649956 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 3 loss : 0.13589516458225262 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 2.1313541666666667 rmse_sparse_metric: 2.1313541666666667 loss_iou : 0.17558348337809246 loss_prec : 0.17756632486979165 loss_loss : 0.1358951695760091 loss_acc : 0.9673453776041666 loss_rec : 0.9840897623697916 val_loss : 2.532514634024766 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.07900547981262207 val_loss_prec : 0.08653933472103542 val_loss_loss : 2.532514360215929 val_loss_acc : 0.8775181240505643 val_loss_rec : 0.7745491663614908 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 4 loss : 0.11389048684442969 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 2.0490625 rmse_sparse_metric: 2.0490625 loss_iou : 0.1996185811360677 loss_prec : 0.20159257253011068 loss_loss : 0.1138906987508138 loss_acc : 0.9730851236979167 loss_rec : 0.9856961059570313 val_loss : 8.58495722938743 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.10192386971579658 val_loss_prec : 0.19616170724232992 val_loss_loss : 8.584956698947483 val_loss_acc : 0.9627514945136176 val_loss_rec : 0.3266310956743028 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 5 loss : 0.09532881293710185 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 2.378229166666667 rmse_sparse_metric: 2.378229166666667 loss_iou : 0.22963691711425782 loss_prec : 0.23156720479329426 loss_loss : 0.09532897313435873 loss_acc : 0.9782980346679687 loss_rec : 0.9877736409505208 val_loss : 10.559455184162491 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.08753267923990886 val_loss_prec : 0.1575686534245809 val_loss_loss : 10.559455023871529 val_loss_acc : 0.9632636176215278 val_loss_rec : 0.26647456487019855 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 6 loss : 0.07562574363061988 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 2.0819791666666667 rmse_sparse_metric: 2.0819791666666667 loss_iou : 0.2568237050374349 loss_prec : 0.25863550821940107 loss_loss : 0.0756257692972819 loss_acc : 0.9828921508789062 loss_rec : 0.9893025716145833 val_loss : 11.668905624705884 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.08269011974334717 val_loss_prec : 0.19104923142327201 val_loss_loss : 11.668904622395834 val_loss_acc : 0.9682659573025174 val_loss_rec : 0.18886461522844103 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 7 loss : 0.06769100399202822 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 2.3123958333333334 rmse_sparse_metric: 2.3123958333333334 loss_iou : 0.27149088541666666 loss_prec : 0.27367930094401044 loss_loss : 0.0676910400390625 loss_acc : 0.9849922688802083 loss_rec : 0.9895144653320312 val_loss : 12.180131924545599 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.08412468433380127 val_loss_prec : 0.15997138288285997 val_loss_loss : 12.180132548014322 val_loss_acc : 0.9604445563422309 val_loss_rec : 0.2399449348449707 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 8 loss : 0.06005031946408053 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 2.2465625 rmse_sparse_metric: 2.2465625 loss_iou : 0.2933336893717448 loss_prec : 0.29533246358235676 loss_loss : 0.06005029678344727 loss_acc : 0.9866059366861979 loss_rec : 0.9910132853190105 val_loss : 14.821350382227036 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.08032339811325073 val_loss_prec : 0.2632046275668674 val_loss_loss : 14.82135009765625 val_loss_acc : 0.9833286073472765 val_loss_rec : 0.14326214790344238 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 9 loss : 0.058349815968667826 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 2.221875 rmse_sparse_metric: 2.221875 loss_iou : 0.3101000722249349 loss_prec : 0.3118464660644531 loss_loss : 0.058349742889404296 loss_acc : 0.9872409057617187 loss_rec : 0.9917488606770833 val_loss : 12.517815000481075 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.07787870698504978 val_loss_prec : 0.2857825491163466 val_loss_loss : 12.517814636230469 val_loss_acc : 0.9768172370062934 val_loss_rec : 0.13944166236453587 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 10 loss : 0.0482115954470828 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 1.9503125 rmse_sparse_metric: 1.9503125 loss_iou : 0.3312089538574219 loss_prec : 0.3328692626953125 loss_loss : 0.04821149190266927 loss_acc : 0.9895101928710938 loss_rec : 0.9922865804036458 val_loss : 13.955339850650894 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.0780107511414422 val_loss_prec : 0.26500195927090114 val_loss_loss : 13.95534176296658 val_loss_acc : 0.9831354353162978 val_loss_rec : 0.13859159416622585 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 11 loss : 0.04933202152824833 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 2.0902083333333334 rmse_sparse_metric: 2.0902083333333334 loss_iou : 0.3328869883219401 loss_prec : 0.33459765116373696 loss_loss : 0.04933205286661784 loss_acc : 0.9893855794270834 loss_rec : 0.9925393676757812 val_loss : 16.757661278049152 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.06531501478619045 val_loss_prec : 0.3991677496168349 val_loss_loss : 16.75765821668837 val_loss_acc : 0.9846226374308268 val_loss_rec : 0.10530128081639607 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 12 loss : 0.04878552877263549 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 2.0572916666666665 rmse_sparse_metric: 2.0572916666666665 loss_iou : 0.34831891377766927 loss_prec : 0.35023468017578124 loss_loss : 0.04878554026285807 loss_acc : 0.9892347208658854 loss_rec : 0.9926947021484375 val_loss : 14.85178346435229 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.07334701220194499 val_loss_prec : 0.3393416934543186 val_loss_loss : 14.851782904730904 val_loss_acc : 0.9831968943277994 val_loss_rec : 0.12560137112935385 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 13 loss : 0.04147944352905446 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 1.975 rmse_sparse_metric: 1.975 loss_iou : 0.3653294626871745 loss_prec : 0.3668243408203125 loss_loss : 0.04147947947184245 loss_acc : 0.9910907999674479 loss_rec : 0.9932733154296876 val_loss : 10.780205345609122 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.08617289861043294 val_loss_prec : 0.17777089277903238 val_loss_loss : 10.780204772949219 val_loss_acc : 0.9674543804592557 val_loss_rec : 0.2760783831278483 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 14 loss : 0.03722713669809006 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 1.8515625 rmse_sparse_metric: 1.8515625 loss_iou : 0.3863360850016276 loss_prec : 0.38789883931477864 loss_loss : 0.0372270933787028 loss_acc : 0.9919660441080729 loss_rec : 0.9940956624348959 val_loss : 15.135421420551008 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.08546278874079387 val_loss_prec : 0.3615957631005181 val_loss_loss : 15.13542005750868 val_loss_acc : 0.9845504760742188 val_loss_rec : 0.13796117570665148 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ... epoch : 15 loss : 0.034510827331791914 a1_sparse_metric: 0.0 abs_rel_sparse_metric: 1.9503125 rmse_sparse_metric: 1.9503125 loss_iou : 0.3962159729003906 loss_prec : 0.3980547078450521 loss_loss : 0.03451083501180013 loss_acc : 0.9925803629557292 loss_rec : 0.9941372680664062 val_loss : 16.985714921520817 val_a1_sparse_metric: 0.0 val_abs_rel_sparse_metric: 79.0 val_rmse_sparse_metric: 79.0 val_loss_iou : 0.0905357797940572 val_loss_prec : 0.2897793451944987 val_loss_loss : 16.985714382595486 val_loss_acc : 0.9830976062350802 val_loss_rec : 0.16758330663045248 - INFO - Saving checkpoint: saved/models/monorec_mask/00/checkpoint.pth ...

opened by fengziyue 0
problem regarding depth_refinement_loss

Hi Felix,

https://github.com/Brummi/MonoRec/blob/b2b3decc130d9ca333d1350096b9838a12f977a3/model/loss_functions/monorec_loss.py#L334 https://github.com/Brummi/MonoRec/blob/b2b3decc130d9ca333d1350096b9838a12f977a3/model/loss_functions/monorec_loss.py#L339 you use the .detach() method in the sdl at depth_refinement_loss, which cut the backpropagation, different from the depth_refinement_loss in the paper. Or did I misunderstand?

opened by bpwl0121 1

MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera

Related tags

Overview

MonoRec

🏗️ ️ Setup

🏃 Running the Example Script

🗃️ Data

KITTI Odometry

Oxford RobotCar

TUM Mono-VO

TUM RGB-D

🏋️ Training & Evaluation

☁️ Pointclouds

Comments

Question 1:

Question 2:

Owner

Felix Wimbauer

A Planar RGB-D SLAM which utilizes Manhattan World structure to provide optimal camera pose trajectory while also providing a sparse reconstruction containing points, lines and planes, and a dense surfel-based reconstruction.

Deep Implicit Moving Least-Squares Functions for 3D Reconstruction

CVPR2022 paper "Dense Learning based Semi-Supervised Object Detection"

OcclusionFusion: realtime dynamic 3D reconstruction based on single-view RGB-D

Keyhole Imaging: Non-Line-of-Sight Imaging and Tracking of Moving Objects Along a Single Optical Path

Fusion-DHL: WiFi, IMU, and Floorplan Fusion for Dense History of Locations in Indoor Environments

Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set (CVPRW 2019). A PyTorch implementation.

Weakly Supervised Dense Event Captioning in Videos, i.e. generating multiple sentence descriptions for a video in a weakly-supervised manner.

Multiple types of NN model optimization environments. It is possible to directly access the host PC GUI and the camera to verify the operation. Intel iHD GPU (iGPU) support. NVIDIA GPU (dGPU) support.

Official pytorch code for SSC-GAN: Semi-Supervised Single-Stage Controllable GANs for Conditional Fine-Grained Image Generation(ICCV 2021)

[TIP 2021] SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

The official implementation code of "PlantStereo: A Stereo Matching Benchmark for Plant Surface Dense Reconstruction."

Learning to Simulate Dynamic Environments with GameGAN (CVPR 2020)

Camera-caps - Examine the camera capabilities for V4l2 cameras

UniMoCo: Unsupervised, Semi-Supervised and Full-Supervised Visual Representation Learning

Project looking into use of autoencoder for semi-supervised learning and comparing data requirements compared to supervised learning.

Hybrid CenterNet - Hybrid-supervised object detection / Weakly semi-supervised object detection

"MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction" (CVPRW 2022) & (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)

TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction