Hi, thanks for publishing this wonderful works.

I want to ask is there any missing about symmetric code. I checked the code but don't see the initialize for symmetric like in LASR source code. Also the results are different from what I expected.

Hi, I'm trying to use Viser or LASR on videos that camera may rotate around the objects. But the results assume the camera is fixed. Have you ever tried videos where the camera has large motions?

I'm currently training VISER on our synthetic dataset. But when we calculate the matching loss in VISER, it always does to pdb in the first iteration. Our input video size is 1024*1024, would this lead to the problem? Also, our dataset has the camera moving between frames. It seems that the main difference between our video and the video in the demo is the resolution.

The pdb sentence is shown as followed. I tested on many videos of our datasets with various camera initial locations and animal actions. Most of them stopped in this sentence within 1 or 2 iterations, others stopped several iterations later. https://github.com/gengshan-y/viser-release/blob/a3943ad80d391f1b60379524de3c5d07f924c6bd/nnutils/mesh_net.py#L806-L808

Hello, I'm trying to run ViSER on some of my own datasets. Out of my 5 datasets, 2 succeed and 3 fail: all with the same failure case:

> /HPS/articulated_nerf/work/viser/nnutils/mesh_net.py(809)forward()
-> self.match_loss = (csm_pred - csm_gt).norm(2,1)[mask].mean() * 0.1
(Pdb) 
Traceback (most recent call last):
  File "optimize.py", line 59, in <module>
    app.run(main)
  File "/HPS/articulated_nerf/work/miniconda3/envs/viser/lib/python3.8/site-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/HPS/articulated_nerf/work/miniconda3/envs/viser/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "optimize.py", line 56, in main
    trainer.train()
  File "/HPS/articulated_nerf/work/viser/nnutils/train_utils.py", line 339, in train
    total_loss,aux_output = self.model(input_batch)
  File "/HPS/articulated_nerf/work/miniconda3/envs/viser/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/HPS/articulated_nerf/work/miniconda3/envs/viser/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 705, in forward
    output = self.module(*inputs[0], **kwargs[0])
  File "/HPS/articulated_nerf/work/miniconda3/envs/viser/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/HPS/articulated_nerf/work/viser/nnutils/mesh_net.py", line 809, in forward
    self.match_loss = (csm_pred - csm_gt).norm(2,1)[mask].mean() * 0.1
  File "/HPS/articulated_nerf/work/viser/nnutils/mesh_net.py", line 809, in forward
    self.match_loss = (csm_pred - csm_gt).norm(2,1)[mask].mean() * 0.1
  File "/HPS/articulated_nerf/work/miniconda3/envs/viser/lib/python3.8/bdb.py", line 88, in trace_dispatch
    return self.dispatch_line(frame)
  File "/HPS/articulated_nerf/work/miniconda3/envs/viser/lib/python3.8/bdb.py", line 113, in dispatch_line
    if self.quitting: raise BdbQuit
bdb.BdbQuit
Traceback (most recent call last):
  File "/HPS/articulated_nerf/work/miniconda3/envs/viser/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/HPS/articulated_nerf/work/miniconda3/envs/viser/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/HPS/articulated_nerf/work/miniconda3/envs/viser/lib/python3.8/site-packages/torch/distributed/launch.py", line 340, in <module>
    main()
  File "/HPS/articulated_nerf/work/miniconda3/envs/viser/lib/python3.8/site-packages/torch/distributed/launch.py", line 326, in main
    sigkill_handler(signal.SIGTERM, None)  # not coming back
  File "/HPS/articulated_nerf/work/miniconda3/envs/viser/lib/python3.8/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
    raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/HPS/articulated_nerf/work/miniconda3/envs/viser/bin/python', '-u', 'optimize.py', '--local_rank=0', '--name=cactus_full-1003-0', '--checkpoint_dir', 'log', '--n_bones', '21', '--num_epochs', '20', '--dataname', 'cactus_full-init', '--ngpu', '1', '--batch_size', '4', '--seed', '1003']' returned non-zero exit status 1.
Killing subprocess 6097

Full error log 1 Full error log 2 Full error log 3

I would tend to assume this is a division by zero in the identified line. Have you encountered this issue before?

I have tried multiple values of init_frame and end_frame for initially optimizing on a subset (where the failure occurs). I have also tried different seed values. I haven't found any choice of these parameters that cause these datasets to avoid this failure case.

Any help or insight you can provide would be appreciated

Hi, Thanks for open-sourcing this awesome work. I tried to run on the dance-twirl video and got the following results:

https://user-images.githubusercontent.com/96632902/152095343-c2fc9026-7835-468a-be92-a2114da35bf1.mp4

It seems the rendered frame does not match the input frame, I organized the data as suggested in the Readme and run by the command sh scripts/dance-twirl.sh and test by bash scripts/render_result.sh dance-twirl log/dance-twirl-1003-ft2/pred_net_latest.pth --catemodel --cnnpp `, could you please let me know if this result is expected or if there's something wrong? Thanks!

Hello,

I would like to ask about the implementation of feature consistency loss in your source code. I was looking through the nnutils/mesh_net.py but could only find match loss and cycle (reprojection) loss, and imatch loss (maybe inverse match loss), which seems not reported in the paper.

Would you please help me locate the implementation of feature consistency loss ?

Thank you in advance.

Hi, thanks for publishing this wonderful works.

I want to ask is it possible for us to use the evalutation code from LASR to evaluate the ViSER's outputs? I think that it is possible, but I don't see you publish the evaluation code. Therefore, I think there is something that I haven't noticed. Thank you very much.

I have a problem in reproducing the ama-female dataset from viser-rlease. I used the init-cam provided by Posenet, I also used your updated ama-female.sh script, and I used the banmo format in vid.py, and changed the smooth loss to the original 0.005. In the end the training of ama-female resulted in an image like this, what could have gone wrong with this process?

I follow the code (https://github.com/gengshan-y/viser-release/blob/main/preprocess/README.md) and run "bash sh scripts/breakdance-flare.sh" But It stopped at the break point