Adversarial Texture Optimization from RGB-D Scans (CVPR 2020).

Related tags

Deep Learning texture reconstruction adversarial 3d-vision cvpr2020
Overview

AdversarialTexture

Adversarial Texture Optimization from RGB-D Scans (CVPR 2020).

AdversarialTexture Teaser

Scanning Data Download

Please refer to data directory for details.

Before run following scripts, please modify the data_path in src/config.py as the absolute path of the data folder (e.g. Adversarial/data) where you download all data.

Prepare for Training (Optimization)

Please refer to src/preprocessing directory for details.

Run Training (Optimization)

Consider execute run_all.sh in parallel.

cd src/textureoptim
python gen_script.py
sh run_all.sh

Result Visualization

The result will be stored in data/result/chairID/chairID.png. You can use them to replace the corresponding default texture in data/shape, and use meshlab to open obj files to see the results.

Alternatively, we provide a simple script to render results. You will be able to see the rendering comparison in data/visual.

cd src
python visualize.py

Authors

© Jingwei Huang, Stanford University

IMPORTANT: If you use this code please cite the following in any resulting publication:

@inproceedings{huang2020adversarial,
  title={Adversarial Texture Optimization from RGB-D Scans},
  author={Huang, Jingwei and Thies, Justus and Dai, Angela and Kundu, Abhijit and Jiang, Chiyu and Guibas, Leonidas J and Niessner, Matthias and Funkhouser, Thomas},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={1559--1568},
  year={2020}
}

The rendering process is a modification of pyRender.

Comments
  • Questions about the code

    Questions about the code

    Dear Jingwei, I have other two questions about the code

    • In Line128 of dataset.py, why 1-uv?

    • In Line 146 of model.py, why discrim_loss is set to 0 when it is greater than gen_loss_GAN?

    opened by raywzy 4
  • What is the effect of Cache?

    What is the effect of Cache?

    Very impressive work! While checking the dataset file, I found you utilize the cache to ensure no repeat calculation for each source image. But it will lead to one-to-one matching during training, rather than selecting the target view randomly. Is it right?

    opened by raywzy 2
  • Cannot download the data

    Cannot download the data

    Hello, I tried to download the data with commands in the readme at data directory, and they all response 404. Is there any thing wrong with the data source? Expect for your reply!

    opened by stark1092 2
  • About custom dataset

    About custom dataset

    Hi, thank you for sharing source code, it's a fancy work!

    I have a question about using custom dasaset. Since I have some RGBD dataset, I composed it to .sens file like BundleFusion (e.g. apt0.sens). But it couldn't work well using render_scan.py.

    Any suggestion or is there a detail about how your .sens file composed?

    Thank you~

    opened by zhongtao93 1
  • Have you ever seen the `134` error code when calling `Sen.Parse` method?

    Have you ever seen the `134` error code when calling `Sen.Parse` method?

    All data is downloaded from your README file.

    And it has already been compiled and passed.

    Sens.Parse(c_char_p(filename.encode('utf8')))

    Process finished with exit code 134
    opened by FavorMylikes 0
  • Versions of dependent packages

    Versions of dependent packages

    Hello,could you write the Dependencies in readme?eg. I install TF2 but the code need TF1.I would be very grateful if you could accept my opinion.Thanks.

    opened by YuzhiChen001 0
  • About Qianyi Zhou's data - the fountain model

    About Qianyi Zhou's data - the fountain model

    It's a great work and thank you for sharing the code!

    I have tried to run the code with your chair data, and it did show a significant improvement compared to L1 result.

    Then, I edit render_scan.py file to read data from Zhou's fountain model (I choose around 30 RGBD frames as key frames, and use code from "Let there be color" to get the obj and mtl files from all 30 key frames.), I run the code with default parameters (λ=10.0, iter=4001).

    But I don't get the fine result as the supplemental shows.

    Here's what I get:

    Before (L1): fountain snapshot00

    After Iterations: fountain snapshot01

    Is anything missing to run such RGBD datasets?

    Thanks again for the interesting work!

    opened by OneEyedEagle 2
Owner
Jingwei Huang
PhD -- Computer Graphics and Vision.
Jingwei Huang
GitHub http://stanford.edu/~jingweih/papers/advtex/
DSAC* for Visual Camera Re-Localization (RGB or RGB-D)

DSAC* for Visual Camera Re-Localization (RGB or RGB-D) Introduction Installation Data Structure Supported Datasets 7Scenes 12Scenes Cambridge Landmark

Visual Learning Lab 143 Dec 22, 2022
Code for the paper: Adversarial Training Against Location-Optimized Adversarial Patches. ECCV-W 2020.

Adversarial Training Against Location-Optimized Adversarial Patches arXiv | Paper | Code | Video | Slides Code for the paper: Sukrut Rao, David Stutz,

Sukrut Rao 32 Dec 13, 2022
Official code for "End-to-End Optimization of Scene Layout" -- including VAE, Diff Render, SPADE for colorization (CVPR 2020 Oral)

End-to-End Optimization of Scene Layout Code release for: End-to-End Optimization of Scene Layout CVPR 2020 (Oral) Project site, Bibtex For help conta

Andrew Luo 41 Dec 9, 2022
Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion (CVPR'2021, Oral)

DSA^2 F: Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion (CVPR'2021, Oral) This repo is the official imp

如今我已剑指天涯 46 Dec 21, 2022
An implementation of "Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport"

Optex An implementation of Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport for TU Delft CS4240. You c

Hans Brouwer 33 Jan 5, 2023
"3D Human Texture Estimation from a Single Image with Transformers", ICCV 2021

Texformer: 3D Human Texture Estimation from a Single Image with Transformers This is the official implementation of "3D Human Texture Estimation from

XiangyuXu 193 Dec 5, 2022
[ICCV'2021] Image Inpainting via Conditional Texture and Structure Dual Generation

[ICCV'2021] Image Inpainting via Conditional Texture and Structure Dual Generation

Xiefan Guo 122 Dec 11, 2022
GANmouflage: 3D Object Nondetection with Texture Fields

GANmouflage: 3D Object Nondetection with Texture Fields Rui Guo1 Jasmine Collins

null 29 Aug 10, 2022
Segmentation and Identification of Vertebrae in CT Scans using CNN, k-means Clustering and k-NN

Segmentation and Identification of Vertebrae in CT Scans using CNN, k-means Clustering and k-NN If you use this code for your research, please cite ou

null 41 Dec 8, 2022
Automatic detection and classification of Covid severity degree in LUS (lung ultrasound) scans

Final-Project Final project in the Technion, Biomedical faculty, by Mor Ventura, Dekel Brav & Omri Magen. Subproject 1: Automatic Detection of LUS Cha

Mor Ventura 1 Dec 18, 2021
Genetic Algorithm, Particle Swarm Optimization, Simulated Annealing, Ant Colony Optimization Algorithm,Immune Algorithm, Artificial Fish Swarm Algorithm, Differential Evolution and TSP(Traveling salesman)

scikit-opt Swarm Intelligence in Python (Genetic Algorithm, Particle Swarm Optimization, Simulated Annealing, Ant Colony Algorithm, Immune Algorithm,A

郭飞 3.7k Jan 3, 2023
library for nonlinear optimization, wrapping many algorithms for global and local, constrained or unconstrained, optimization

NLopt is a library for nonlinear local and global optimization, for functions with and without gradient information. It is designed as a simple, unifi

Steven G. Johnson 1.4k Dec 25, 2022
Racing line optimization algorithm in python that uses Particle Swarm Optimization.

Racing Line Optimization with PSO This repository contains a racing line optimization algorithm in python that uses Particle Swarm Optimization. Requi

Parsa Dahesh 6 Dec 14, 2022
PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

pytorch-a2c-ppo-acktr Update (April 12th, 2021) PPO is great, but Soft Actor Critic can be better for many continuous control tasks. Please check out

Ilya Kostrikov 3k Jan 9, 2023
FedMM: Saddle Point Optimization for Federated Adversarial Domain Adaptation

This repository contains the code accompanying the paper " FedMM: Saddle Point Optimization for Federated Adversarial Domain Adaptation" Paper link: R

null 20 Jun 29, 2022
PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

Ilya Kostrikov 3k Dec 31, 2022
Code for "Adversarial Attack Generation Empowered by Min-Max Optimization", NeurIPS 2021

Min-Max Adversarial Attacks [Paper] [arXiv] [Video] [Slide] Adversarial Attack Generation Empowered by Min-Max Optimization Jingkang Wang, Tianyun Zha

Jingkang Wang 12 Nov 23, 2022
UDP++ (ECCVW 2020 Oral), (Winner of COCO 2020 Keypoint Challenge).

UDP-Pose This is the pytorch implementation for UDP++, which won the Fisrt place in COCO Keypoint Challenge at ECCV 2020 Workshop. Top-Down Results on

null 20 Jul 29, 2022
Defending graph neural networks against adversarial attacks (NeurIPS 2020)

GNNGuard: Defending Graph Neural Networks against Adversarial Attacks Authors: Xiang Zhang ([email protected]), Marinka Zitnik (marinka@hms.

Zitnik Lab @ Harvard 44 Dec 7, 2022