Sketch-Based 3D Exploration with Stacked Generative Adversarial Networks

Overview

pix2vox

[Demonstration video]
Sketch-Based 3D Exploration with Stacked Generative Adversarial Networks.

Generated samples

Single-category generation

Multi-category generation

Requirements

The following python packages are required for running the application. If you are using anaconda, you can easily install VTK5 and PyQt4 (or they may already be installed).

$ conda install -c anaconda vtk=5.10.1
$ conda install -c anaconda pyqt=4.11.4
$ pip install qdarkstyle

Getting started

  1. Install the python packages above
  2. Install Git LSF (You need Git LSF to download trained model file)
  3. Download the code from GitHub:
$ git lsf clone https://github.com/maxorange/pix2vox.git
$ cd pix2vox
  1. Run the code:
$ python application.py
Comments
  • Out of range: Read less bytes than requested

    Out of range: Read less bytes than requested

    Hello,

    This work is very interesting! However, when I run the code ("python application.py"), it gets some errors like "Out of range: Read less bytes than requested", caused by "model = sgan.Model("params/sgan_model.ckpt")". Is the params model given right?

    Thanks.

    opened by dongdu3 3
  • Failed to load module

    Failed to load module "canberra-gtk-module"

    Failed to load module "canberra-gtk-module" /home/duan/anaconda3/envs/venv/lib/python2.7/site-packages/qdarkstyle/init.py:181: FutureWarning: load_stylesheet() will not receive pyside parameter in version 3. Set QtPy environment variable to specify the Qt binding insteady. FutureWarning 2018-10-31 19:36:32.905967: E tensorflow/stream_executor/cuda/cuda_dnn.cc:343] Loaded runtime CuDNN library: 7.1.4 but source was compiled with: 7.2.1. CuDNN library major and minor version needs to match or have higher minor version in case of CuDNN 7.0 or later version. If using a binary install, upgrade your CuDNN library. If building from sources, make sure the library loaded at runtime is compatible with the version specified during compile configuration.

    my platform :Ubuntu18.04+anaconda+Tensorflow-gpu+cuda9.0+cudnn7.1.4 how can I solve these problems?Thanks!

    opened by qingchunlizhi 1
  • model/sgan.py

    model/sgan.py", line 154, in edge c = conv2d(edge, [4, 4, nc, nf], 'h1', bias=True) NameError: name 'conv2d' is not defined

    i modified the script to run on PyQt5 instead of PyQt4 my TensorFlow version is 1.1.0 inside python 3.6, however i am encountering the bellow issue: any suggestions will be appreciated

    `python application.py /home/ahmed/anaconda3/envs/py361/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /home/ahmed/anaconda3/envs/py361/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /home/ahmed/anaconda3/envs/py361/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /home/ahmed/anaconda3/envs/py361/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /home/ahmed/anaconda3/envs/py361/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /home/ahmed/anaconda3/envs/py361/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) /home/ahmed/anaconda3/envs/py361/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /home/ahmed/anaconda3/envs/py361/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /home/ahmed/anaconda3/envs/py361/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /home/ahmed/anaconda3/envs/py361/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /home/ahmed/anaconda3/envs/py361/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /home/ahmed/anaconda3/envs/py361/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) WARNING:tensorflow:From /home/ahmed/Documents/3DAI/pix2vox_maxorange/model/sgan.py:13: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

    WARNING:tensorflow:From /home/ahmed/Documents/3DAI/pix2vox_maxorange/model/sgan.py:15: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

    2020-09-04 17:35:49.652680: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA 2020-09-04 17:35:49.675335: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3493490000 Hz 2020-09-04 17:35:49.676389: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55d9a11d3230 executing computations on platform Host. Devices: 2020-09-04 17:35:49.676412: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): , WARNING:tensorflow:From /home/ahmed/Documents/3DAI/pix2vox_maxorange/model/sgan.py:19: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

    WARNING:tensorflow:From /home/ahmed/Documents/3DAI/pix2vox_maxorange/model/sgan.py:153: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

    Traceback (most recent call last): File "application.py", line 12, in model = sgan.Model("params/sgan_model.ckpt") File "/home/ahmed/Documents/3DAI/pix2vox_maxorange/model/sgan.py", line 16, in init self.build_model(model_path) File "/home/ahmed/Documents/3DAI/pix2vox_maxorange/model/sgan.py", line 28, in build_model edge = enc.edge(self.edge, self.train, self.n_cls) File "/home/ahmed/Documents/3DAI/pix2vox_maxorange/model/sgan.py", line 154, in edge c = conv2d(edge, [4, 4, nc, nf], 'h1', bias=True) NameError: name 'conv2d' is not defined `

    opened by ahmedshingaly 0
  • Fail to get Voxel model

    Fail to get Voxel model

    The problem I am getting is I don't get the reconstruction in my output window. It is to be noted that drawing and painting are working properly. but corresponding 3D model is not generated. provide some help!

    opened by wizard499 1
  • VTK Version 5.10.1 not available.

    VTK Version 5.10.1 not available.

    Can't find vtk 5.10.1 on https://vtk.org/download/ as well as on the web. Using VTK version 8 is leading to many backward incompatible errors. Any suggestions please.

    opened by kunalkhadilkar 3
  • TypeError: 'NoneType' object is not iterable

    TypeError: 'NoneType' object is not iterable

    /home/duan/anaconda3/envs/venv/lib/python2.7/site-packages/qdarkstyle/init.py:181: FutureWarning: load_stylesheet() will not receive pyside parameter in version 3. Set QtPy environment variable to specify the Qt binding insteady. FutureWarning Traceback (most recent call last): File "/home/duan/pix2vox/ui/gui_main.py", line 115, in slider.valueChanged.connect(lambda value, j=int(index): self.opt_engine.set_label(float(value)/100, j)) File "/home/duan/pix2vox/opt/constrained_opt.py", line 59, in set_label self.update_voxel_model() File "/home/duan/pix2vox/opt/constrained_opt.py", line 42, in update_voxel_model self.preprocess_constraints() File "/home/duan/pix2vox/opt/constrained_opt.py", line 35, in preprocess_constraints color_old, edge_old = self.constraints TypeError: 'NoneType' object is not iterable

    I don‘t know whether it is caused by the data?

    opened by qingchunlizhi 7
  • OutOfRangeError (see above for traceback): Read less bytes than requested

    OutOfRangeError (see above for traceback): Read less bytes than requested

    Thanks for your awesome code share!

    Recently I do some research on 3d construction and find your great code in github,but when I run the demo,some error occur.just like #2,all the output is here:

    OutOfRangeError (see above for traceback): Read less bytes than requested
    	 [[Node: save/RestoreV2_53 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_53/tensor_names, save/RestoreV2_53/shape_and_slices)]]
    
    

    I can find sgan_model.ckpt.data-00000-of-00001 ,sgan_model.ckpt.meta and sgan_model.ckpt.index in param folder,but the size of them is very small:

    4.0K	sgan_model.ckpt.data-00000-of-00001
    216K	sgan_model.ckpt.meta
    4.0K	sgan_model.ckpt.index
    

    I run the code in Ubuntu 14.04 with tensorflow 1.5.0 in Python3.4,I solve all the dependent libraries and grammar conflict between py2 and py3 ,but still fail to run the code.It is indeed double happiness for me to get your help,thanks a lot!

    opened by qixuxiang 3
Owner
Takumi Moriya
Takumi Moriya
Generative Exploration and Exploitation - This is an improved version of GENE.

GENE This is an improved version of GENE. In the original version, the states are generated from the decoder of VAE. We have to check whether the gere

null 33 Mar 23, 2022
Semi-supervised Representation Learning for Remote Sensing Image Classification Based on Generative Adversarial Networks

SSRL-for-image-classification Semi-supervised Representation Learning for Remote Sensing Image Classification Based on Generative Adversarial Networks

Feng 2 Nov 19, 2021
Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning Source Code

Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning Source Code

STARS Laboratory 8 Sep 14, 2022
A web-based application for quick, scalable, and automated hyperparameter tuning and stacked ensembling in Python.

Xcessiv Xcessiv is a tool to help you create the biggest, craziest, and most excessive stacked ensembles you can think of. Stacked ensembles are simpl

Reiichiro Nakano 1.3k Nov 17, 2022
Grow Function: Generate 3D Stacked Bifurcating Double Deep Cellular Automata based organisms which differentiate using a Genetic Algorithm...

Grow Function: A 3D Stacked Bifurcating Double Deep Cellular Automata which differentiates using a Genetic Algorithm... TLDR;High Def Trees that you can mint as NFTs on Solana

Nathaniel Gibson 4 Oct 8, 2022
PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence) and pre-trained model on ImageNet dataset

Reference-Based-Sketch-Image-Colorization-ImageNet This is a PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization usin

Yuzhi ZHAO 11 Jul 28, 2022
Minimal PyTorch implementation of Generative Latent Optimization from the paper "Optimizing the Latent Space of Generative Networks"

Minimal PyTorch implementation of Generative Latent Optimization This is a reimplementation of the paper Piotr Bojanowski, Armand Joulin, David Lopez-

Thomas Neumann 117 Nov 27, 2022
StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation.

StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation.

null 3k Jan 8, 2023
[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

Large Scale Image Completion via Co-Modulated Generative Adversarial Networks, ICLR 2021 (Spotlight) Demo | Paper [NEW!] Time to play with our interac

Shengyu Zhao 373 Jan 2, 2023
Regularizing Generative Adversarial Networks under Limited Data (CVPR 2021)

Regularizing Generative Adversarial Networks under Limited Data [Project Page][Paper] Implementation for our GAN regularization method. The proposed r

Google 148 Nov 18, 2022
NR-GAN: Noise Robust Generative Adversarial Networks

NR-GAN: Noise Robust Generative Adversarial Networks (CVPR 2020) This repository provides PyTorch implementation for noise robust GAN (NR-GAN). NR-GAN

Takuhiro Kaneko 59 Dec 11, 2022
HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis Jungil Kong, Jaehyeon Kim, Jaekyoung Bae In our paper, we p

Rishikesh (ऋषिकेश) 31 Dec 8, 2022
Generating Anime Images by Implementing Deep Convolutional Generative Adversarial Networks paper

AnimeGAN - Deep Convolutional Generative Adverserial Network PyTorch implementation of DCGAN introduced in the paper: Unsupervised Representation Lear

Rohit Kukreja 23 Jul 21, 2022
π-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis

π-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis Project Page | Paper | Data Eric Ryan Chan*, Marco Monteiro*, Pe

null 375 Dec 31, 2022
Unofficial implementation of Alias-Free Generative Adversarial Networks. (https://arxiv.org/abs/2106.12423) in PyTorch

alias-free-gan-pytorch Unofficial implementation of Alias-Free Generative Adversarial Networks. (https://arxiv.org/abs/2106.12423) This implementation

Kim Seonghyeon 502 Jan 3, 2023
PyTorch implementations of Generative Adversarial Networks.

This repository has gone stale as I unfortunately do not have the time to maintain it anymore. If you would like to continue the development of it as

Erik Linder-Norén 13.4k Jan 8, 2023
Image Deblurring using Generative Adversarial Networks

DeblurGAN arXiv Paper Version Pytorch implementation of the paper DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks. Our netwo

Orest Kupyn 2.2k Jan 1, 2023
Code for the paper "TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks"

TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks This is a Python3 / Pytorch implementation of TadGAN paper. The associated

Arun 92 Dec 3, 2022
Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary Differential Equations

ODE GAN (Prototype) in PyTorch Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary

Somshubra Majumdar 15 Feb 10, 2022