A PyTorch implementation of "DGC-Net: Dense Geometric Correspondence Network"

Overview

DGC-Net: Dense Geometric Correspondence Network

This is a PyTorch implementation of our work "DGC-Net: Dense Geometric Correspondence Network"

TL;DR A CNN-based approach to obtain dense pixel correspondences between two views.

License

Shield: CC BY-NC-SA 4.0

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, available only for non-commercial use.

CC BY-NC-SA 4.0

Installation

  • create and activate conda environment with Python 3.x
conda create -n my_fancy_env python=3.7
source activate my_fancy_env
  • install Pytorch v1.0.0 and torchvision library
pip install torch torchvision
  • install all dependencies by running the following command:
pip install -r requirements.txt

Getting started

  • eval.py demonstrates the results on the HPatches dataset To be able to run eval.py script:

    • Download an archive with pre-trained models click and extract it to the project folder
    • Download HPatches dataset (Full image sequences). The dataset is available here at the end of the page
    • Run the following command:
    python eval.py --image-data-path /path/to/hpatches-geometry
    
  • train.py is a script to train DGC-Net/DGCM-Net model from scratch. To run this script, please follow the next procedure:

    python train.py --image-data-path /path/to/TokyoTimeMachine
    

Performance on HPatches dataset

Method / HPatches ID Viewpoint 1 Viewpoint 2 Viewpoint 3 Viewpoint 4 Viewpoint 5
PWC-Net 4.43 11.44 15.47 20.17 28.30
GM best model 9.59 18.55 21.15 27.83 35.19
DGC-Net (paper) 1.55 5.53 8.98 11.66 16.70
DGCM-Net (paper) 2.97 6.85 9.95 12.87 19.13
DGC-Net (repo) 1.74 5.88 9.07 12.14 16.50
DGCM-Net (repo) 2.33 5.62 9.55 11.59 16.48

Note: There is a difference in numbers presented in the original paper and obtained by the models of this repo. It might be related to the fact that both models (DGC-Net and DGCM-Net) have been trained using Pytorch v0.3.

More qualitative results are presented on the project page

How to cite

If you use this software in your own research, please cite our publication:

@inproceedings{Melekhov+Tiulpin+Sattler+Pollefeys+Rahtu+Kannala:2018,
      title = {{DGC-Net}: Dense geometric correspondence network},
      author = {Melekhov, Iaroslav and Tiulpin, Aleksei and 
               Sattler, Torsten, and 
               Pollefeys, Marc and 
               Rahtu, Esa and Kannala, Juho},
       year = {2019},
       booktitle = {Proceedings of the IEEE Winter Conference on 
                    Applications of Computer Vision (WACV)}
}
Comments
  • torch.jit.frontend.NotSupportedError

    torch.jit.frontend.NotSupportedError

    Is it Python2 or pytorch version specific?

    python3 eval.py --image-data-path ~/hpatches-sequences-release Traceback (most recent call last): File "eval.py", line 8, in import torchvision.transforms as transforms File "/usr/local/lib/python3.6/dist-packages/torchvision/init.py", line 1, in from torchvision import models File "/usr/local/lib/python3.6/dist-packages/torchvision/models/init.py", line 12, in from . import detection File "/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/init.py", line 1, in from .faster_rcnn import * File "/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/faster_rcnn.py", line 13, in from .rpn import AnchorGenerator, RPNHead, RegionProposalNetwork File "/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/rpn.py", line 8, in from . import _utils as det_utils File "/usr/local/lib/python3.6/dist-packages/torchvision/models/detection/_utils.py", line 74, in @torch.jit.script File "/usr/local/lib/python3.6/dist-packages/torch/jit/init.py", line 364, in script graph = _script_graph(fn, _frames_up=_frames_up + 1) File "/usr/local/lib/python3.6/dist-packages/torch/jit/init.py", line 359, in _script_graph ast = get_jit_ast(fn) File "/usr/local/lib/python3.6/dist-packages/torch/jit/frontend.py", line 132, in get_jit_ast return build_def(SourceRangeFactory(source), py_ast.body[0]) File "/usr/local/lib/python3.6/dist-packages/torch/jit/frontend.py", line 151, in build_def build_stmts(ctx, body)) File "/usr/local/lib/python3.6/dist-packages/torch/jit/frontend.py", line 123, in build_stmts stmts = [build_stmt(ctx, s) for s in stmts] File "/usr/local/lib/python3.6/dist-packages/torch/jit/frontend.py", line 123, in stmts = [build_stmt(ctx, s) for s in stmts] File "/usr/local/lib/python3.6/dist-packages/torch/jit/frontend.py", line 140, in call return method(ctx, node) File "/usr/local/lib/python3.6/dist-packages/torch/jit/frontend.py", line 205, in build_Assign rhs = build_expr(ctx, stmt.value) File "/usr/local/lib/python3.6/dist-packages/torch/jit/frontend.py", line 140, in call return method(ctx, node) File "/usr/local/lib/python3.6/dist-packages/torch/jit/frontend.py", line 314, in build_Call func = build_expr(ctx, expr.func) File "/usr/local/lib/python3.6/dist-packages/torch/jit/frontend.py", line 140, in call return method(ctx, node) File "/usr/local/lib/python3.6/dist-packages/torch/jit/frontend.py", line 300, in build_Attribute value = build_expr(ctx, expr.value) File "/usr/local/lib/python3.6/dist-packages/torch/jit/frontend.py", line 140, in call return method(ctx, node) File "/usr/local/lib/python3.6/dist-packages/torch/jit/frontend.py", line 422, in build_Subscript raise NotSupportedError(base.range(), "slicing multiple dimensions at the same time isn't supported yet") torch.jit.frontend.NotSupportedError: slicing multiple dimensions at the same time isn't supported yet proposals (Tensor): boxes to be encoded """

    # perform some unpacking to make it JIT-fusion friendly
    wx = weights[0]
    wy = weights[1]
    ww = weights[2]
    wh = weights[3]
    
    proposals_x1 = proposals[:, 0].unsqueeze(1)
                   ~~~~~~~~~ <--- HERE
    proposals_y1 = proposals[:, 1].unsqueeze(1)
    proposals_x2 = proposals[:, 2].unsqueeze(1)
    proposals_y2 = proposals[:, 3].unsqueeze(1)
    
    reference_boxes_x1 = reference_boxes[:, 0].unsqueeze(1)
    reference_boxes_y1 = reference_boxes[:, 1].unsqueeze(1)
    reference_boxes_x2 = reference_boxes[:, 2].unsqueeze(1)
    reference_boxes_y2 = reference_boxes[:, 3].unsqueeze(1)
    
    opened by deepdad 3
  • Bump opencv-python from 4.0.0.21 to 4.1.1.26

    Bump opencv-python from 4.0.0.21 to 4.1.1.26

    Bumps opencv-python from 4.0.0.21 to 4.1.1.26.

    Release notes

    Sourced from opencv-python's releases.

    4.1.1.26

    OpenCV version 4.1.1.

    Changes:

    • FFmpeg has been compiled with https support on Linux builds #229
    • CI build logic related changes #197, #227, #228
    • Custom libjepg-turbo removed because it's provided by OpenCV #231
    • 64-bit Qt builds are now smaller #236
    • Custom builds should be now rather easy to do locally #235:
      1. Clone this repository
      2. Optional: set up ENABLE_CONTRIB and ENABLE_HEADLESS environment variables to 1 if needed
      3. Optional: add additional Cmake arguments to CMAKE_ARGS environment variable
      4. Run python setup.py bdist_wheel

    4.1.0.25

    OpenCV version 4.1.0.

    4.0.1.24

    OpenCV version 4.0.1.

    • macOS build fix and migration to Travis Xcode 8.3 environment (#182)
    • typo fix (#169)
    • README updated with further instructions related to Windows DLL issues (thanks to #185 & #170 for bringing this to my attention)

    There were issues with deployment with the previous tags 22 & 23. For 23 there exists a release in PyPI, but macOS wheels are not included. Please ignore those releases.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • 0%

    0%

    DGC-Net$ python3.7 eval.py --image-data-path ~/hpatches-sequences-release 0%| | 0/59 [00:00<?, ?it/s]torch.Size([1, 225, 15, 15])

    It outputs a torch.size with 0% progress bar by running the first test from the Readme. Probably not quite right, is it? It reported

    GC-Net$ python3.7 eval.py --image-data-path ~/hpatches-sequences-release 0%| | 0/59 [00:00<?, ?it/s]torch.Size([1, 225, 15, 15]) Traceback (most recent call last): File "eval.py", line 93, in device) File "/home/ubuntu/sticker_detection/DGC-Net/utils/evaluate.py", line 61, in calculate_epe_hpatches estimates_grid, estimates_mask = net(source_img, target_img) File "/home/ubuntu/anaconda3/envs/match/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, **kwargs) File "/home/ubuntu/anaconda3/envs/match/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward return self.module(*inputs[0], **kwargs[0]) File "/home/ubuntu/anaconda3/envs/match/lib/python3.7/site-packages/torch/nn/modules/module.py", line 541, in call result = self.forward(*input, **kwargs) File "/home/ubuntu/sticker_detection/DGC-Net/model/net.py", line 68, in forward sys.exit() NameError: name 'sys' is not defined

    before, which I fixed by importing sys obviously.

    bug 
    opened by deepdad 1
  • Script for demo

    Script for demo

    Hi man, great job! Do you have some script for demo? I mean, the code for getting as input two images (source and target) and generating at the output the transformed image? If so, could you please provide it? Thanks!

    opened by marcelo-mendonca 1
  • Bump opencv-python from 4.0.0.21 to 4.2.0.32

    Bump opencv-python from 4.0.0.21 to 4.2.0.32

    Bumps opencv-python from 4.0.0.21 to 4.2.0.32.

    Release notes

    Sourced from opencv-python's releases.

    4.2.0.32

    OpenCV version 4.2.0.

    Changes:

    • macOS environment updated from xcode8.3 to xcode 9.4
    • macOS uses now Qt 5 instead of Qt 4
    • Nasm version updated to Docker containers
    • multibuild updated

    Fixes:

    • don't use deprecated brew tap-pin, instead refer to the full package name when installing #267
    • replace get_config_var() with get_config_vars() in setup.py #274
    • add workaround for DLL errors in Windows Server #264

    4.1.2.30

    OpenCV version 4.1.2.

    Changes:

    • Python 3.8 builds added to the build matrix
    • Support for Python 3.4 builds dropped (Python 3.4 is in EOL)
    • multibuild updated
    • minor build logic changes
    • Docker images rebuilt

    Notes:

    Please note that Python 2.7 enters into EOL phase in January 2020. opencv-python Python 2.7 wheels won't be provided after that.

    4.1.1.26

    OpenCV version 4.1.1.

    Changes:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Homography Augmentation

    Homography Augmentation

    Hi Melekhov. I find that the H parameters are fixed in the csv file. I'm interested in the homography augmentation parameters you used. Could you please provide some guidelines?

    opened by drinkingcoder 0
  • Application for sequences of images than just two images

    Application for sequences of images than just two images

    Apologies if my question sounds dumb!

    I would like to know if it makes sense to use netvlad/dgc-net for place recognition of problems dealing with sequences of images (SLAM) over seasonal changes (no specific viewpoint changes)?

    What I have in mind is Nordland dataset in which images rather suffer from illumination changes over seasons.

    Cheers,

    opened by mrgransky 0
  • one question ...

    one question ...

    model/net.py

     for k in reversed(range(4)):  ####3,2,1,0
            p1, p2 = target_pyr[k], source_pyr[k]
            est_map = F.interpolate(input=estimates_grid[-1], scale_factor=2, mode='bilinear', align_corners=False)
    
            p1_w = F.grid_sample(p1, est_map.transpose(1,2).transpose(2,3))
            est_map = self.__dict__['_modules']['reg_' + str(k)](x1=p1_w, x2=p2, x3=est_map)
            estimates_grid.append(est_map)
    

    In my opinion :

       p1, p2 = target_pyr[k], source_pyr[k]
    

    should be: p1,p2 = source_pyr[k],target_pyr[k]

    is it right? if i was wrong. pls explain it briefly. I'm confused.... thanks very much!!

    Additional, I think your work is outstanding .Maybe my graduation project will be closely relate to your excellent work. So I 'd like to learn about any details.

    Forgive me for my poor English can’t fully express the feeling of gratitude.

    opened by liyangszu3d 1
  • i can't get  tokyo time machine dataset......

    i can't get tokyo time machine dataset......

    Thanks for your excellent work!!!!

    I'm very interested in your research,but i can't get the tokyo time machine dataset with your links....

    maybe my question is native,,but i'm eager to get your reply!

    Datasets Place recognition datasets: Tokyo Time Machine: available on request. The train/val splits are provided with our code. Tokyo 24/7: available on request here. Pittsburgh 250k: available on request here. The train/val/test splits are provided with our code. Tiny subset of Tokyo Time Machine (21 MB). Contains 360 images, just to be used to validate if the NetVLAD code is set up correctly.

    opened by liyangszu3d 3
Owner
null
An essential implementation of BYOL in PyTorch + PyTorch Lightning

Essential BYOL A simple and complete implementation of Bootstrap your own latent: A new approach to self-supervised Learning in PyTorch + PyTorch Ligh

Enrico Fini 48 Sep 27, 2022
RealFormer-Pytorch Implementation of RealFormer using pytorch

RealFormer-Pytorch Implementation of RealFormer using pytorch. Includes comparison with classical Transformer on image classification task (ViT) wrt C

Simo Ryu 89 Sep 11, 2022
A PyTorch implementation of the paper Mixup: Beyond Empirical Risk Minimization in PyTorch

Mixup: Beyond Empirical Risk Minimization in PyTorch This is an unofficial PyTorch implementation of mixup: Beyond Empirical Risk Minimization. The co

Harry Yang 121 Aug 29, 2022
A pytorch implementation of Pytorch-Sketch-RNN

Pytorch-Sketch-RNN A pytorch implementation of https://arxiv.org/abs/1704.03477 In order to draw other things than cats, you will find more drawing da

Alexis David Jacq 169 Jul 28, 2022
PyTorch implementation of Advantage async actor-critic Algorithms (A3C) in PyTorch

Advantage async actor-critic Algorithms (A3C) in PyTorch @inproceedings{mnih2016asynchronous, title={Asynchronous methods for deep reinforcement lea

LEI TAI 111 Mar 27, 2022
Pytorch-diffusion - A basic PyTorch implementation of 'Denoising Diffusion Probabilistic Models'

PyTorch implementation of 'Denoising Diffusion Probabilistic Models' This reposi

Arthur Juliani 48 Sep 18, 2022
Fang Zhonghao 11 Aug 7, 2022
RETRO-pytorch - Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

RETRO - Pytorch (wip) Implementation of RETRO, Deepmind's Retrieval based Attent

Phil Wang 441 Sep 23, 2022
HashNeRF-pytorch - Pure PyTorch Implementation of NVIDIA paper on Instant Training of Neural Graphics primitives

HashNeRF-pytorch Instant-NGP recently introduced a Multi-resolution Hash Encodin

Yash Sanjay Bhalgat 509 Sep 24, 2022
Generic template to bootstrap your PyTorch project with PyTorch Lightning, Hydra, W&B, and DVC.

NN Template Generic template to bootstrap your PyTorch project. Click on Use this Template and avoid writing boilerplate code for: PyTorch Lightning,

Luca Moschella 495 Sep 15, 2022
A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

This repository holds NVIDIA-maintained utilities to streamline mixed precision and distributed training in Pytorch. Some of the code here will be included in upstream Pytorch eventually. The intention of Apex is to make up-to-date utilities available to users as quickly as possible.

NVIDIA Corporation 6.6k Sep 25, 2022
Objective of the repository is to learn and build machine learning models using Pytorch. 30DaysofML Using Pytorch

30 Days Of Machine Learning Using Pytorch Objective of the repository is to learn and build machine learning models using Pytorch. List of Algorithms

Mayur 117 Jul 29, 2022
Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch

Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch

Pytorch Lightning 1.4k Sep 22, 2022
Amazon Forest Computer Vision: Satellite Image tagging code using PyTorch / Keras with lots of PyTorch tricks

Amazon Forest Computer Vision Satellite Image tagging code using PyTorch / Keras Here is a sample of images we had to work with Source: https://www.ka

Mamy Ratsimbazafy 359 Aug 27, 2022
The Incredible PyTorch: a curated list of tutorials, papers, projects, communities and more relating to PyTorch.

This is a curated list of tutorials, projects, libraries, videos, papers, books and anything related to the incredible PyTorch. Feel free to make a pu

Ritchie Ng 9.1k Sep 28, 2022
Amazon Forest Computer Vision: Satellite Image tagging code using PyTorch / Keras with lots of PyTorch tricks

Amazon Forest Computer Vision Satellite Image tagging code using PyTorch / Keras Here is a sample of images we had to work with Source: https://www.ka

Mamy Ratsimbazafy 359 Aug 27, 2022
A bunch of random PyTorch models using PyTorch's C++ frontend

PyTorch Deep Learning Models using the C++ frontend Gettting started Clone the repo 1. https://github.com/mrdvince/pytorchcpp 2. cd fashionmnist or

Vince 0 Jul 13, 2021
PyTorch Autoencoders - Implementing a Variational Autoencoder (VAE) Series in Pytorch.

PyTorch Autoencoders Implementing a Variational Autoencoder (VAE) Series in Pytorch. Inspired by this repository Model List check model paper conferen

Subin An 7 Oct 20, 2021
PyTorch-LIT is the Lite Inference Toolkit (LIT) for PyTorch which focuses on easy and fast inference of large models on end-devices.

PyTorch-LIT PyTorch-LIT is the Lite Inference Toolkit (LIT) for PyTorch which focuses on easy and fast inference of large models on end-devices. With

Amin Rezaei 154 Sep 13, 2022