SurfEmb (CVPR 2022) - SurfEmb: Dense and Continuous Correspondence Distributions

Overview

SurfEmb

SurfEmb: Dense and Continuous Correspondence Distributions
for Object Pose Estimation with Learnt Surface Embeddings

Rasmus Laurvig Haugard, Anders Glent Buch
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022
pre-print | project-site

The easiest way to explore correspondence distributions is through the project site.

The following describes how to reproduce the results.

Install

Download surfemb:

$ git clone https://github.com/rasmushaugaard/surfemb.git
$ cd surfemb

All following commands are expected to be run in the project root directory.

Install conda , create a new environment, surfemb, and activate it:

$ conda env create -f environment.yml
$ conda activate surfemb

Download BOP data

Download and extract datasets from the BOP site. Base archive, and object models are needed for both training and inference. For training, PBR-BlenderProc4BOP training images are needed as well, and for inference, the BOP'19/20 test images are needed.

Extract the datasets under data/bop (or make a symbolic link).

Model

Download a trained model (see releases):

$ wget https://github.com/rasmushaugaard/surfemb/releases/download/v0.0.1/tless-2rs64lwh.compact.ckpt -P data/models

OR

Train a model:

$ python -m surfemb.scripts.train [dataset] --gpus [gpu ids]

For example, to train a model on T-LESS on cuda:0

$ python -m surfemb.scripts.train tless --gpus 0

Inference data

We use the detections from CosyPose's MaskRCNN models, and sample surface points evenly for inference.
For ease of use, this data can be downloaded and extracted as follows:

$ wget https://github.com/rasmushaugaard/surfemb/releases/download/v0.0.1/inference_data.zip
$ unzip inference_data.zip

OR

Extract detections and sample surface points

Surface samples

First, flip the normals of ITODD object 18, which is inside out.

Then remove invisible parts of the objects

$ python -m surfemb.scripts.misc.surface_samples_remesh_visible [dataset] 

sample points evenly from the mesh surface

$ python -m surfemb.scripts.misc.surface_samples_sample_even [dataset] 

and recover the normals for the sampled points.

$ python -m surfemb.scripts.misc.surface_samples_recover_normals [dataset] 

Detection results

Download CosyPose in the same directory as SurfEmb was downloaded in, install CosyPose and follow their guide to download their BOP-trained detection results. Then:

$ python -m surfemb.scripts.misc.load_detection_results [dataset]

Inference inspection

To see pose estimation examples on the training images run

$ python -m surfemb.scripts.infer_debug [model_path] --device [device]

[device] could for example be cuda:0 or cpu.

Add --real to use the test images with simulated crops based on the ground truth poses, or further add --detections to use the CosyPose detections.

Inference for BOP evaluation

Inference is run on the (real) test images with CosyPose detections:

$ python -m surfemb.scripts.infer [model_path] --device [device]

Pose estimation results are saved to data/results.
To obtain results with depth (requires running normal inference first), run

$ python -m surfemb.scripts.infer_refine_depth [model_path] --device [device]

The results can be formatted for BOP evaluation using

$ python -m surfemb.scripts.misc.format_results_for_eval [poses_path]

Either upload the formatted results to the BOP Challenge website or evaluate using the BOP toolkit.

Extra

Custom dataset: Format the dataset as a BOP dataset and put it in data/bop.

Comments
  • Custom dataset

    Custom dataset

    If I want to train my datasets(texture-less), which has 4 classes.

    First, how should I train the 2D detector? And MaskRCNN or Retinanet?

    Second, in surfemb, What parameters should I modify?

    Can you give me some advice? Thanks~

    opened by woodcore-an 25
  • Question about inference data

    Question about inference data

    For some strange reason wget does not download data to the correct place on my machine, so I downloaded the inference_data.zip file manually. I am now unsure where to extract its contents? Would I do this in the root directory or perhaps in /data or /data/models. Thank you.

    opened by meropis 9
  • Some problems encountered when training tless

    Some problems encountered when training tless

    I downloaded the tless on bop and put it under data/bop/tless, the code can load the cad models , i use python -m surfemb.scripts.train tless --gpus 0 to run, but it runs to After trainer.fit, the following error will occur: TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found <class 'trimesh.caching.TrackedArray'> Sorry, am I setting it wrong? image

    opened by transcend-lzy 6
  • train on a custom dataset

    train on a custom dataset

    I'm trying to train surfemb on a custom dataset. But when I do inference, I find some files/dirs are musts(they're detection_results, surface_samples, surface_samples_normals). For dectection_results, I've already known it's from CosyPose, but I don't know how to generate it in detail. For surface_samples, I followed your guidance that run command $ python -m surfemb.scripts.misc.surface_samples_remesh_visible clip first. But I encountered this error:

    Traceback (most recent call last):
      File "/root/miniconda3/envs/surfemb/lib/python3.8/runpy.py", line 194, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/root/miniconda3/envs/surfemb/lib/python3.8/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/root/surfemb/surfemb/scripts/misc/surface_samples_remesh_visible.py", line 32, in <module>
        ms.compute_scalar_ambient_occlusion(occmode='per-Face (deprecated)', reqviews=256)
    AttributeError: 'pymeshlab.pmeshlab.MeshSet' object has no attribute 'compute_scalar_ambient_occlusion'
    

    Could you please give me any suggestions on these two problems?

    opened by LeroyChou 5
  • question about pose score.

    question about pose score.

    Hi, thank you again.

    I'm comparing the pose score you proposed in paper and implemented in codes such that I encounter some questions.

    1. In this line you calculate neg_mask_log_prob by inversing mask_lgts before feeding it into the logsigmoid. why do you inverse mask_lgts? Does it actually mean anything?
    2. Can I think of the pose score as confidence as long as I map its value to [0, 1] via a kind of mono-increasing function?
    opened by LeroyChou 4
  • scores in resulst and bop19_average_recall

    scores in resulst and bop19_average_recall

    Question 1:I used ycbv-jwpvdij1.compact.ckpt(a trained model that you provided) to infer test datasets in ycbv(python -m surfemb.scripts.infer), then python -m surfemb.scripts.misc.format_results_for_eval, the score in results all is negative, for example,-0.339 , -0.401.Is that normal? image A:scene_id B:img_id C:est_obj_id D: score.

    opened by cats0212 4
  • TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

    TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

    when i want to train the tless python -m surfemb.scripts.train tless:

    Traceback (most recent call last):
      File "/home/zzz/miniconda3/envs/surfemb/lib/python3.8/runpy.py", line 194, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "/home/zzz/miniconda3/envs/surfemb/lib/python3.8/runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "/home/zzz/github/surfemb/surfemb/scripts/train.py", line 121, in <module>
        main()
      File "/home/zzz/github/surfemb/surfemb/scripts/train.py", line 61, in main
        model = SurfaceEmbeddingModel(n_objs=len(obj_ids), **vars(args))
      File "/home/zzz/github/surfemb/surfemb/surface_embedding.py", line 48, in __init__
        n_class=(emb_dim + 1) if separate_decoders else n_objs * (emb_dim + 1),
    TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'
    

    It Seems the emb_dim is None. But it has the default value emb_dim=12.

    opened by woodcore-an 4
  • 2080ti one gpu gives out of memory error while training

    2080ti one gpu gives out of memory error while training

    Hi, thanks for your great work, I hope I will make it work and be able to use it on my custom dataset. my problem is this; I have one 2080ti and I am trying to train the tless pbr dataset but I get an error "cuda out of memory" . I have used smaller batch size which is 8, I have decreased the number of workers to 0. but it keeps giving the error ( ok now it gives the error later than before but it still gives the error)

    it only works if I decrease the scenes from 50 to 1 in train_pbr folder. otherwise no chance.

    is this normal behavior with this one gpu , or I am missing something

    thanks in advance Screenshot from 2022-06-13 11-00-50

    opened by smoothumut 3
  • cannot import name 'egl' from 'glcontext'

    cannot import name 'egl' from 'glcontext'

    I have tried to run the Inference Inspection code in my windows machine with the given inference data as proposed in the README but I got the error:

    $ python -m surfemb.scripts.infer_debug data/models/tless-2rs64lwh.compact.ckpt --device cpu
    loading objects: 0it [00:00, ?it/s]
    Traceback (most recent call last):
      File "C:\Users\39331\anaconda3\envs\surfemb\lib\runpy.py", line 194, in _run_module_as_main
        return _run_code(code, main_globals, None,
      File "C:\Users\39331\anaconda3\envs\surfemb\lib\runpy.py", line 87, in _run_code
        exec(code, run_globals)
      File "C:\Users\39331\Documenti\Final Year Project\surfemb\surfemb\scripts\infer_debug.py", line 43, in <module>
        renderer = ObjCoordRenderer(objs, res_crop)
      File "C:\Users\39331\Documenti\Final Year Project\surfemb\surfemb\data\renderer.py", line 43, in __init__
        self.ctx = moderngl.create_context(standalone=True, backend='egl', device_index=device_idx)
      File "C:\Users\39331\anaconda3\envs\surfemb\lib\site-packages\moderngl\context.py", line 1619, in create_context
        ctx.mglo, ctx.version_code = mgl.create_context(glversion=require, mode=mode, **settings)
      File "C:\Users\39331\anaconda3\envs\surfemb\lib\site-packages\glcontext\__init__.py", line 49, in get_backend_by_name
        return _egl()
      File "C:\Users\39331\anaconda3\envs\surfemb\lib\site-packages\glcontext\__init__.py", line 106, in _egl
        from glcontext import egl
    ImportError: cannot import name 'egl' from 'glcontext' (C:\Users\39331\anaconda3\envs\surfemb\lib\site-packages\glcontext\__init__.py)
    

    I have tried installing OpenGL again but it did not solve the problem, I cannot find any sources for solving the dependency. How would you suggest me to solve it?

    opened by cuccomat 3
  • Wait at 0% while training.Epoch 0:       0%              0/13000

    Wait at 0% while training.Epoch 0: 0% 0/13000

    Hi, i run 'python -m surfemb.scripts.train ycbv', and ./data/bop/ycbv/models have 21 .ply files, and 80 folder in ./data/bop/ycbv/train_real. I did not use synth imgs, but the program always 0%. Is the program preprocessing image information,crop object from img?I waited a dozen hours and it was still 0%. like Epoch 0: 0%. The python is still running.Do I have to wait a long time before train model?Are you in a similar situation?

    But if I just have 3 .ply files in ./data/bop/ycba/models,and 1 folder in ./data/bop/ycbv/train_real,it will soon(3 - 4 minutes) train cnn .Finally, the trained model is obtained. For example, obj_000008.ply, obj_000014.ply, obj_000021.ply in ./data/bop/ycba/models, 000000 in ./data/bop/ycbv/train_real(imgs in 000000 only have 3 types of objects, obj_8, obj_14, obj_21).

    If I want to train all the objects in ycbv at once, Do I have to wait longer?I'm using a server, CPU performance is not weak.

    { "os": "Linux-4.15.0-175-generic-x86_64-with-debian-buster-sid", "python": "3.7.11", "heartbeatAt": "2022-04-09T09:34:05.311715", "startedAt": "2022-04-09T09:34:02.496814", "docker": null, "gpu": "GeForce RTX 3090", "gpu_count": 8, "cpu_count": 40, "cuda": null, "args": [], "state": "running", "program": "-m surfemb.scripts.train", "git": { "remote": "https://github.com/rasmushaugaard/surfemb.git", "commit": "46f46ddc5670848d696968dc8ec65c8ce62b16a8" }, "email": "[email protected]", "root": "/home/aa/prjs/surfemb", "host": "sddx-PR4908P", "username": "aa", "executable": "/home/aa/anaconda3/envs/d2_1.10/bin/python" }

    logs: 2022-04-09 11:02:46,213 INFO MainThread:16337 [wandb_setup.py:_flush():75] Loading settings from /home/aa/.config/wandb/settings 2022-04-09 11:02:46,214 INFO MainThread:16337 [wandb_setup.py:_flush():75] Loading settings from /home/aa/prjs/bcnet/pose/surfemb/wandb/settings 2022-04-09 11:02:46,214 INFO MainThread:16337 [wandb_setup.py:_flush():75] Loading settings from environment variables: {'api_key': 'REDACTED', 'mode': 'offline', '_require_service': 'True'} 2022-04-09 11:02:46,214 WARNING MainThread:16337 [wandb_setup.py:_flush():75] Could not find program at -m surfemb.scripts.train 2022-04-09 11:02:46,214 INFO MainThread:16337 [wandb_setup.py:_flush():75] Inferring run settings from compute environment: {'program_relpath': None, 'program': '-m surfemb.scripts.train'} 2022-04-09 11:02:46,214 INFO MainThread:16337 [wandb_init.py:_log_setup():405] Logging user logs to /home/aa/prjs/bcnet/pose/surfemb/wandb/offline-run-20220409_110246-3fewafz3/logs/debug.log 2022-04-09 11:02:46,214 INFO MainThread:16337 [wandb_init.py:_log_setup():406] Logging internal logs to /home/aa/prjs/bcnet/pose/surfemb/wandb/offline-run-20220409_110246-3fewafz3/logs/debug-internal.log 2022-04-09 11:02:46,215 INFO MainThread:16337 [wandb_init.py:init():439] calling init triggers 2022-04-09 11:02:46,215 INFO MainThread:16337 [wandb_init.py:init():443] wandb.init called with sweep_config: {} config: {} 2022-04-09 11:02:46,215 INFO MainThread:16337 [wandb_init.py:init():492] starting backend 2022-04-09 11:02:46,228 INFO MainThread:16337 [backend.py:_multiprocessing_setup():101] multiprocessing start_methods=fork,spawn,forkserver, using: spawn 2022-04-09 11:02:46,232 INFO MainThread:16337 [wandb_init.py:init():501] backend started and connected 2022-04-09 11:02:46,238 INFO MainThread:16337 [wandb_init.py:init():565] updated telemetry 2022-04-09 11:02:46,578 INFO MainThread:16337 [wandb_init.py:init():625] starting run threads in backend 2022-04-09 11:02:49,104 INFO MainThread:16337 [wandb_run.py:_console_start():1733] atexit reg 2022-04-09 11:02:49,106 INFO MainThread:16337 [wandb_run.py:_redirect():1606] redirect: SettingsConsole.WRAP 2022-04-09 11:02:49,107 INFO MainThread:16337 [wandb_run.py:_redirect():1643] Wrapping output streams. 2022-04-09 11:02:49,108 INFO MainThread:16337 [wandb_run.py:_redirect():1667] Redirects installed. 2022-04-09 11:02:49,109 INFO MainThread:16337 [wandb_init.py:init():664] run started, returning control to user process 2022-04-09 11:02:49,130 INFO MainThread:16337 [wandb_run.py:_config_callback():992] config_cb None None {'n_objs': 21, 'emb_dim': 12, 'n_pos': 1024, 'n_neg': 1024, 'lr_cnn': 0.0003, 'lr_mlp': 3e-05, 'mlp_name': 'siren', 'mlp_hidden_features': 256, 'mlp_hidden_layers': 2, 'key_noise': 0.001, 'warmup_steps': 2000, 'separate_decoders': True, 'pa_sigma': 0.0, 'align_corners': False, 'dataset': 'ycbv', 'n_valid': 200, 'res_data': 256, 'res_crop': 224, 'batch_size': 16, 'num_workers': 'None', 'min_visib_fract': 0.1, 'max_steps': 500000, 'gpus': 2, 'debug': False, 'ckpt': 'None', 'synth': False, 'real': True} 2022-04-09 11:07:50,141 WARNING MsgRouterThr:16337 [router.py:message_loop():76] message_loop has been closed

    opened by cats0212 3
  • question about bbox and mask

    question about bbox and mask

    Thanks again for this great work. I am still trying to make it work for our case. In order to run the model with our custom dataset, should we need to have bbox and mask in our bop formatted custom dataset? we can provide bbox_visible but not bbox. and we can provide mask_visible but not mask? how important are they? would be bbox_visible and mask_visible enough? if we need them for this model, do you have any idea how to generate that ? thanks thanks in advance

    opened by smoothumut 2
  • Number of threads explodes when training

    Number of threads explodes when training

    In the training script, an additional environment variable needs to be set in the worker_init_fn function

    os.environ['OMP_NUM_THREADS'] = 1

    See torch comment for additional information

    opened by nikwoj 0
  • Pose Refiner Diverges

    Pose Refiner Diverges

    Thank you for the wonderful work. Both the paper and the code are a pleasure to read.

    I have tried the approach on a different dataset and would like to ask for your expert opinion, if I may. A fraction of the predictions (~60%) are very good even with only RGB refinement, but the remaining pose predictions are far away from the actual pose (about 1m in l1 distance) and could be ruled out by calculating the xyz boundaries of the crop. The input pose from the PNP between both fraction almost equally good.

    • Do you have an idea how to discipline the refinement?
    • How would you analyze the quality of the incoming query image or the sampled keys?
    • Do you have any other suggestions what to look for in these cases?

    Thanks again for the wonderful work.

    opened by FabianSchuetze 4
  • error when resuming from checkpoint

    error when resuming from checkpoint

    whenever i try to resume from a previous checkpoint, i get this error: File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/moritz/surfemb/surfemb/surfemb/scripts/train.py", line 123, in <module> main() File "/home/moritz/surfemb/surfemb/surfemb/scripts/train.py", line 119, in main trainer.fit(model, loader_train, loader_valid) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 768, in fit self._call_and_handle_interrupt( File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 721, in _call_and_handle_interrupt return trainer_fn(*args, **kwargs) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 809, in _fit_impl results = self._run(model, ckpt_path=self.ckpt_path) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1234, in _run results = self._run_stage() File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1321, in _run_stage return self._run_train() File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1351, in _run_train self.fit_loop.run() File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 204, in run self.advance(*args, **kwargs) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/loops/fit_loop.py", line 268, in advance self._outputs = self.epoch_loop.run(self._data_fetcher) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 204, in run self.advance(*args, **kwargs) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 208, in advance batch_output = self.batch_loop.run(batch, batch_idx) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 204, in run self.advance(*args, **kwargs) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 88, in advance outputs = self.optimizer_loop.run(split_batch, optimizers, batch_idx) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 204, in run self.advance(*args, **kwargs) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 203, in advance result = self._run_optimization( File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 256, in _run_optimization self._optimizer_step(optimizer, opt_idx, batch_idx, closure) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 369, in _optimizer_step self.trainer._call_lightning_module_hook( File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1593, in _call_lightning_module_hook output = fn(*args, **kwargs) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/core/lightning.py", line 1644, in optimizer_step optimizer.step(closure=optimizer_closure) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/core/optimizer.py", line 168, in step step_output = self._strategy.optimizer_step(self._optimizer, self._optimizer_idx, closure, **kwargs) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/strategies/strategy.py", line 193, in optimizer_step return self.precision_plugin.optimizer_step(model, optimizer, opt_idx, closure, **kwargs) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/pytorch_lightning/plugins/precision/precision_plugin.py", line 155, in optimizer_step return optimizer.step(closure=closure, **kwargs) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/torch/optim/lr_scheduler.py", line 65, in wrapper return wrapped(*args, **kwargs) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/torch/optim/optimizer.py", line 109, in wrapper return func(*args, **kwargs) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/torch/optim/adam.py", line 157, in step adam(params_with_grad, File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/torch/optim/adam.py", line 213, in adam func(params, File "/home/moritz/anaconda3/envs/surfemb/lib/python3.8/site-packages/torch/optim/adam.py", line 255, in _single_tensor_adam assert not step_t.is_cuda, "If capturable=False, state_steps should not be CUDA tensors." AssertionError: If capturable=False, state_steps should not be CUDA tensors.

    Any idea how to resolve this?

    also i cant get the training to run with standard settings. i get an outofmemory error on a rtx3070ti (8gb) if i dont run at n-valid = 2 and batch-size = 1

    opened by MoritzkoLP 0
  • 'ambient_occlusion' filter throws core dumped for tless models

    'ambient_occlusion' filter throws core dumped for tless models

    When I run the surface_samples_remesh_visible script for tless objects the processing fails with

    Aborted (core dumped)

    did anyone face the same issue or did I miss some steps before running the script?

    opened by praveen5733 1
A PyTorch implementation of "DGC-Net: Dense Geometric Correspondence Network"

DGC-Net: Dense Geometric Correspondence Network This is a PyTorch implementation of our work "DGC-Net: Dense Geometric Correspondence Network" TL;DR A

null 191 Dec 16, 2022
MLOps will help you to understand how to build a Continuous Integration and Continuous Delivery pipeline for an ML/AI project.

page_type languages products description sample python azure azure-machine-learning-service azure-devops Code which demonstrates how to set up and ope

null 1 Nov 1, 2021
[CVPR 2022] CoTTA Code for our CVPR 2022 paper Continual Test-Time Domain Adaptation

CoTTA Code for our CVPR 2022 paper Continual Test-Time Domain Adaptation Prerequisite Please create and activate the following conda envrionment. To r

Qin Wang 87 Jan 8, 2023
This is the code for the paper "Jinkai Zheng, Xinchen Liu, Wu Liu, Lingxiao He, Chenggang Yan, Tao Mei: Gait Recognition in the Wild with Dense 3D Representations and A Benchmark. (CVPR 2022)"

Gait3D-Benchmark This is the code for the paper "Jinkai Zheng, Xinchen Liu, Wu Liu, Lingxiao He, Chenggang Yan, Tao Mei: Gait Recognition in the Wild

null 82 Jan 4, 2023
CVPR '21: In the light of feature distributions: Moment matching for Neural Style Transfer

In the light of feature distributions: Moment matching for Neural Style Transfer (CVPR 2021) This repository provides code to recreate results present

Nikolai Kalischek 49 Oct 13, 2022
An official implementation of "SFNet: Learning Object-aware Semantic Correspondence" (CVPR 2019, TPAMI 2020) in PyTorch.

PyTorch implementation of SFNet This is the implementation of the paper "SFNet: Learning Object-aware Semantic Correspondence". For more information,

CV Lab @ Yonsei University 87 Dec 30, 2022
Official repository for Few-shot Image Generation via Cross-domain Correspondence (CVPR '21)

Few-shot Image Generation via Cross-domain Correspondence Utkarsh Ojha, Yijun Li, Jingwan Lu, Alexei A. Efros, Yong Jae Lee, Eli Shechtman, Richard Zh

Utkarsh Ojha 251 Dec 11, 2022
Learning Correspondence from the Cycle-consistency of Time (CVPR 2019)

TimeCycle Code for Learning Correspondence from the Cycle-consistency of Time (CVPR 2019, Oral). The code is developed based on the PyTorch framework,

Xiaolong Wang 706 Nov 29, 2022
Imposter-detector-2022 - HackED 2022 Team 3IQ - 2022 Imposter Detector

HackED 2022 Team 3IQ - 2022 Imposter Detector By Aneeljyot Alagh, Curtis Kan, Jo

Joshua Ji 3 Aug 20, 2022
The 7th edition of NTIRE: New Trends in Image Restoration and Enhancement workshop will be held on June 2022 in conjunction with CVPR 2022.

NTIRE 2022 - Image Inpainting Challenge Important dates 2022.02.01: Release of train data (input and output images) and validation data (only input) 2

Andrés Romero 37 Nov 27, 2022
[CVPR'21] FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space

FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space by Quande Liu, Cheng Chen, Ji

Quande Liu 178 Jan 6, 2023
Dense Contrastive Learning (DenseCL) for self-supervised representation learning, CVPR 2021.

Dense Contrastive Learning for Self-Supervised Visual Pre-Training This project hosts the code for implementing the DenseCL algorithm for se

Xinlong Wang 491 Jan 3, 2023
Quasi-Dense Similarity Learning for Multiple Object Tracking, CVPR 2021 (Oral)

Quasi-Dense Tracking This is the offical implementation of paper Quasi-Dense Similarity Learning for Multiple Object Tracking. We present a trailer th

ETH VIS Research Group 327 Dec 27, 2022
Tensorflow implementation of the paper "HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences", CVPR 2021.

HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences Tensorflow implementation of the paper "HumanGPS: Geodesic PreServing Feature fo

Google Interns 50 Dec 21, 2022
Universal Probability Distributions with Optimal Transport and Convex Optimization

Sylvester normalizing flows for variational inference Pytorch implementation of Sylvester normalizing flows, based on our paper: Sylvester normalizing

Rianne van den Berg 172 Dec 13, 2022
DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

The Official PyTorch Implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

Shiyi Lan 3 Oct 15, 2021
The PyTorch implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision.

DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision The PyTorch implementation of DiscoBox: Weakly Supe

Shiyi Lan 1 Oct 23, 2021
Pytorch implementation of Generative Models as Distributions of Functions 🌿

Generative Models as Distributions of Functions This repo contains code to reproduce all experiments in Generative Models as Distributions of Function

Emilien Dupont 117 Dec 29, 2022
HiddenMarkovModel implements hidden Markov models with Gaussian mixtures as distributions on top of TensorFlow

Class HiddenMarkovModel HiddenMarkovModel implements hidden Markov models with Gaussian mixtures as distributions on top of TensorFlow 2.0 Installatio

Susara Thenuwara 2 Nov 3, 2021