D²Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos

Overview

D²Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos

This repository contains the implementation for "D²Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos" by Christian Schmidt, Ali Athar, Sabarinath Mahadevan, and Bastian Leibe (Accepted to WACV 2022).

Code coming soon.

You might also like...
This is an official implementation of CvT: Introducing Convolutions to Vision Transformers.

Introduction This is an official implementation of CvT: Introducing Convolutions to Vision Transformers. We present a new architecture, named Convolut

Implements an infinite sum of poisson-weighted convolutions

An infinite sum of Poisson-weighted convolutions Kyle Cranmer, Aug 2018 If viewing on GitHub, this looks better with nbviewer: click here Consider a v

PyTorch implementation of the R2Plus1D convolution based ResNet architecture described in the paper "A Closer Look at Spatiotemporal Convolutions for Action Recognition"

R2Plus1D-PyTorch PyTorch implementation of the R2Plus1D convolution based ResNet architecture described in the paper "A Closer Look at Spatiotemporal

RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition

RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition (PyTorch) Paper: https://arxiv.org/abs/2105.01883 Citation: @

[ICCV 2021] Official Tensorflow Implementation for
[ICCV 2021] Official Tensorflow Implementation for "Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions"

KPAC: Kernel-Sharing Parallel Atrous Convolutional block This repository contains the official Tensorflow implementation of the following paper: Singl

an implementation of Revisiting Adaptive Convolutions for Video Frame Interpolation using PyTorch
an implementation of Revisiting Adaptive Convolutions for Video Frame Interpolation using PyTorch

revisiting-sepconv This is a reference implementation of Revisiting Adaptive Convolutions for Video Frame Interpolation [1] using PyTorch. Given two f

Unofficial pytorch implementation of 'Image Inpainting for Irregular Holes Using Partial Convolutions'
Unofficial pytorch implementation of 'Image Inpainting for Irregular Holes Using Partial Convolutions'

pytorch-inpainting-with-partial-conv Official implementation is released by the authors. Note that this is an ongoing re-implementation and I cannot f

Kernel Point Convolutions
Kernel Point Convolutions

Created by Hugues THOMAS Introduction Update 27/04/2020: New PyTorch implementation available. With SemanticKitti, and Windows supported. This reposit

Simple Tensorflow implementation of
Simple Tensorflow implementation of "Adaptive Convolutions for Structure-Aware Style Transfer" (CVPR 2021)

AdaConv — Simple TensorFlow Implementation [Paper] : Adaptive Convolutions for Structure-Aware Style Transfer (CVPR 2021) Note This repository does no

Comments
  • About the backbone pretrained weights from facebookresearch/VMZ

    About the backbone pretrained weights from facebookresearch/VMZ

    I follow the instruction to download the backbone files in https://github.com/facebookresearch/VMZ/blob/main/c2/tutorials/model_zoo.md, but the snapshot file of this site is '.pkl' file and your weights path in config/model/backbone/irCSN152.yaml is a '.pth' file,

    then:

    Traceback (most recent call last):
    
      File "/media/wilson/SD_Data/DE/video_shadow/d2conv3d-main/main.py", line 51, in main
        model = SaliencyModel(cfg)
      File "/media/wilson/SD_Data/DE/video_shadow/d2conv3d-main/experiments/saliency.py", line 24, in __init__
        self.model = build_network(hparams.model)
      File "/media/wilson/SD_Data/DE/video_shadow/d2conv3d-main/utils/construct.py", line 44, in build_network
        return retrieve("network", cfg.name)(cfg)
      File "/media/wilson/SD_Data/DE/video_shadow/d2conv3d-main/network/segmentation/encoder_decoder.py", line 17, in __init__
        backbone = build_backbone(cfg.backbone)
      File "/media/wilson/SD_Data/DE/video_shadow/d2conv3d-main/utils/construct.py", line 62, in build_backbone
        load_weights(
      File "/media/wilson/SD_Data/DE/video_shadow/d2conv3d-main/utils/util.py", line 144, in load_weights
        state_dict = torch.load(weights_path, map_location="cpu", encoding="utf-8")
      File "/home/wilson/anaconda3/envs/d2conv3d/lib/python3.8/site-packages/torch/serialization.py", line 713, in load
        return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
      File "/home/wilson/anaconda3/envs/d2conv3d/lib/python3.8/site-packages/torch/serialization.py", line 920, in _legacy_load
        magic_number = pickle_module.load(f, **pickle_load_args)
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xde in position 0: invalid continuation byte
    
    Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
    

    and I have tried to revise the line 84 in utils/util.py:

    state_dict = torch.load(weights_path, map_location="cpu")
    to
    state_dict = torch.load(weights_path, map_location="cpu",, encoding="latin1")
    
    

    something wrong:

    Traceback (most recent call last):
      File "/media/wilson/SD_Data/DE/video_shadow/d2conv3d-main/main.py", line 51, in main
        model = SaliencyModel(cfg)
      File "/media/wilson/SD_Data/DE/video_shadow/d2conv3d-main/experiments/saliency.py", line 24, in __init__
        self.model = build_network(hparams.model)
      File "/media/wilson/SD_Data/DE/video_shadow/d2conv3d-main/utils/construct.py", line 44, in build_network
        return retrieve("network", cfg.name)(cfg)
      File "/media/wilson/SD_Data/DE/video_shadow/d2conv3d-main/network/segmentation/encoder_decoder.py", line 17, in __init__
        backbone = build_backbone(cfg.backbone)
      File "/media/wilson/SD_Data/DE/video_shadow/d2conv3d-main/utils/construct.py", line 62, in build_backbone
        load_weights(
      File "/media/wilson/SD_Data/DE/video_shadow/d2conv3d-main/utils/util.py", line 144, in load_weights
        state_dict = torch.load(weights_path, map_location="cpu", encoding="latin1")
      File "/home/wilson/anaconda3/envs/d2conv3d/lib/python3.8/site-packages/torch/serialization.py", line 713, in load
        return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
      File "/home/wilson/anaconda3/envs/d2conv3d/lib/python3.8/site-packages/torch/serialization.py", line 922, in _legacy_load
        raise RuntimeError("Invalid magic number; corrupt file?")
    RuntimeError: Invalid magic number; corrupt file?
    
    Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
    

    It seems can be only read by

    with open(weights_path, "rb") as f:
        caffe2_checkpoint = pickle.load(f, encoding="latin1")
    

    How can I use this '.pkl' file?

    opened by WilsonAir 2
  • Unable to train with DDP

    Unable to train with DDP

    Thanks for your excellent work! However, when I tried to train the model, the error occurs as shown below.

    I follow the instruction in README.md to build the environment with docker nvidia/cuda:11.1-cudnn8-devel.

    image

    Thanks for your time to solve my problem!

    opened by johnson111788 1
Owner
null
Official code for "Stereo Waterdrop Removal with Row-wise Dilated Attention (IROS2021)"

Stereo-Waterdrop-Removal-with-Row-wise-Dilated-Attention This repository includes official codes for "Stereo Waterdrop Removal with Row-wise Dilated A

null 29 Oct 1, 2022
PyTorch version repo for CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes

Study-CSRNet-pytorch This is the PyTorch version repo for CSRNet: Dilated Convolutional Neural Networks for Understanding the Highly Congested Scenes

null 0 Mar 1, 2022
TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction

TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction TSDF++ is a novel multi-object TSDF formulation that can encode mult

ETHZ ASL 130 Dec 29, 2022
BMW TechOffice MUNICH 148 Dec 21, 2022
Dynamic View Synthesis from Dynamic Monocular Video

Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer This repository contains code to compute depth from a

Intelligent Systems Lab Org 2.3k Jan 1, 2023
Dynamic View Synthesis from Dynamic Monocular Video

Dynamic View Synthesis from Dynamic Monocular Video Project Website | Video | Paper Dynamic View Synthesis from Dynamic Monocular Video Chen Gao, Ayus

Chen Gao 139 Dec 28, 2022
Dynamic vae - Dynamic VAE algorithm is used for anomaly detection of battery data

Dynamic VAE frame Automatic feature extraction can be achieved by probability di

null 10 Oct 7, 2022
PyTorch Implementation of CvT: Introducing Convolutions to Vision Transformers

CvT: Introducing Convolutions to Vision Transformers Pytorch implementation of CvT: Introducing Convolutions to Vision Transformers Usage: img = torch

Rishikesh (ऋषिकेश) 193 Jan 3, 2023
This is an official implementation of CvT: Introducing Convolutions to Vision Transformers.

Introduction This is an official implementation of CvT: Introducing Convolutions to Vision Transformers. We present a new architecture, named Convolut

Microsoft 408 Dec 30, 2022
Implementations of orthogonal and semi-orthogonal convolutions in the Fourier domain with applications to adversarial robustness

Orthogonalizing Convolutional Layers with the Cayley Transform This repository contains implementations and source code to reproduce experiments for t

CMU Locus Lab 36 Dec 30, 2022