Image Completion with Deep Learning in TensorFlow

Overview

Image Completion with Deep Learning in TensorFlow

Citations

Please consider citing this project in your publications if it helps your research. The following is a BibTeX and plaintext reference. The BibTeX entry requires the url LaTeX package.

@misc{amos2016image,
    title        = {{Image Completion with Deep Learning in TensorFlow}},
    author       = {Amos, Brandon},
    howpublished = {\url{http://bamos.github.io/2016/08/09/deep-completion}},
    note         = {Accessed: [Insert date here]}
}

Brandon Amos. Image Completion with Deep Learning in TensorFlow.
http://bamos.github.io/2016/08/09/deep-completion.
Accessed: [Insert date here]
Comments
  • ValueError: Variable d_bn1/d_bn1_2/moments/moments_1/mean/ExponentialMovingAverage/ does not exist

    ValueError: Variable d_bn1/d_bn1_2/moments/moments_1/mean/ExponentialMovingAverage/ does not exist

    when i train model ,errors happened ,...please help me

    Traceback (most recent call last): File "./train-dcgan.py", line 39, in is_crop=False, checkpoint_dir=FLAGS.checkpoint_dir) File "/home/page/wp/dcgan-completion.tensorflow/model.py", line 65, in init self.build_model() File "/home/page/wp/dcgan-completion.tensorflow/model.py", line 81, in build_model self.D_, self.D_logits_ = self.discriminator(self.G, reuse=False) File "/home/page/wp/dcgan-completion.tensorflow/model.py", line 312, in discriminator h1 = lrelu(self.d_bn1(conv2d(h0, self.df_dim*2, name='d_h1_conv'))) File "/home/page/wp/dcgan-completion.tensorflow/ops.py", line 34, in call ema_apply_op = self.ema.apply([batch_mean, batch_var]) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/moving_averages.py", line 375, in apply colocate_with_primary=(var.op.type in ["Variable", "VariableV2"])) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/slot_creator.py", line 123, in create_zeros_slot colocate_with_primary=colocate_with_primary) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/slot_creator.py", line 103, in create_slot return _create_slot_var(primary, val, '') File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/slot_creator.py", line 55, in _create_slot_var slot = variable_scope.get_variable(scope, initializer=val, trainable=False) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 988, in get_variable custom_getter=custom_getter) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 890, in get_variable custom_getter=custom_getter) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 348, in get_variable validate_shape=validate_shape) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 333, in _true_getter caching_device=caching_device, validate_shape=validate_shape) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 657, in _get_single_variable "VarScope?" % name) ValueError: Variable d_bn1/d_bn1_2/moments/moments_1/mean/ExponentialMovingAverage/ does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?

    opened by pageedward 5
  • add scratch var scope with reuse==False

    add scratch var scope with reuse==False

    Just started with tensorflow yesterday, so my confidence that this is the right/best fix is low. However, the build_model() step would not complete pre-commit, and now it does.

    I'm opening this PR so I can get feedback. Thanks!

    https://github.com/bamos/dcgan-completion.tensorflow/issues/14

    opened by shoshber 5
  • some  problem

    some problem

    Sorry to trouble you again,I use 64*64 grayscale images of Chinese characters and trying to repair Chinese characters.But the completed and hats_imgs seem to no change . When training after a period of training the g_loss is a lot of bigger than d_loss, when I use your method to repair, the loss is very high,I have tried to change learning rate lam momentum .the loss is still very high about 7500,can you help me 。I hope you can understand what I mean, my English is not good.

    opened by myw8 5
  • Fixes image_size parameter for training, works for powers of 2 >= 8

    Fixes image_size parameter for training, works for powers of 2 >= 8

    I was not able to get the image_size parameter working when training, it seemed like the code had some hardcoded assumptions about the image size in the generator.

    This PR fixes that, but only for cases where the image is square with side lengths that are a power of 2, that is 8 pixels or more.

    The generator is parameterised based on the image size, with more upscale/transpose convolutions for more powers of 2.

    My assumption is that larger images require more complex models in the discriminator. However, I have not yet found a way to effectively parameterise the discriminator to scale with image size. For now it still works OK for me up to 128, which is as high as I can do with my limited GPU memory.

    Other small changes:

    • Removes the hardcoded requirement for png images to be input. Maybe don't accept this PR if you don't want people's existing command line args to break (now just uses the glob, rather than accepting a directory).
    • Fixes directory creation on python 2.7
    • Deleted one or two lines of commented code.
    • Turned data shuffling back on.

    I won't be offended if you don't accept this PR :) I'm not even close to being an expert. Thank you so much for the original code!

    opened by md5wasp 4
  • error while loading the pretrained model

    error while loading the pretrained model

    when i run the demo using the pretrained model ,errors happened ,the log says the graph do not include d_bn1/moving_mean,i check the graph by using "print graph.get_operations()",and do not find d_bn1.i am not sure what happended...please help me

    Traceback (most recent call last): File "complete.py", line 35, in dcgan.complete(args) File "/home/oeasy/pobei/dcgan-completion.tensorflow-master/model.py", line 225, in complete isLoaded = self.load(self.checkpoint_dir) File "/home/oeasy/pobei/dcgan-completion.tensorflow-master/model.py", line 376, in load self.saver.restore(self.sess, ckpt.model_checkpoint_path) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1388, in restore {self.saver_def.filename_tensor_name: save_path}) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 766, in run run_metadata_ptr) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 964, in _run feed_dict_string, options, run_metadata) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1014, in _do_run target_list, options, run_metadata) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1034, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.NotFoundError: Tensor name "d_bn1/moving_variance" not found in checkpoint files checkpoint/DCGAN.model-106502 [[Node: save/RestoreV2_3 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_3/tensor_names, save/RestoreV2_3/shape_and_slices)]]

    Caused by op u'save/RestoreV2_3', defined at: File "complete.py", line 34, in checkpoint_dir=args.checkpointDir, lam=args.lam) File "/home/oeasy/pobei/dcgan-completion.tensorflow-master/model.py", line 65, in init self.build_model() File "/home/oeasy/pobei/dcgan-completion.tensorflow-master/model.py", line 110, in build_model self.saver = tf.train.Saver(max_to_keep=1) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1000, in init self.build() File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1030, in build restore_sequentially=self._restore_sequentially) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 624, in build restore_sequentially, reshape) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 361, in _AddRestoreOps tensors = self.restore_op(filename_tensor, saveable, preferred_shard) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 200, in restore_op [spec.tensor.dtype])[0]) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_io_ops.py", line 441, in restore_v2 dtypes=dtypes, name=name) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2240, in create_op original_op=self._default_original_op, op_def=op_def) File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1128, in init self._traceback = _extract_stack()

    NotFoundError (see above for traceback): Tensor name "d_bn1/moving_variance" not found in checkpoint files checkpoint/DCGAN.model-106502 [[Node: save/RestoreV2_3 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_3/tensor_names, save/RestoreV2_3/shape_and_slices)]]

    opened by pobei 4
  • makedirs() got an unexpected keyword argument 'exist_ok

    makedirs() got an unexpected keyword argument 'exist_ok

    Hi I have completed the training process, but for the complete.py file I got this Error

    File "./complete.py", line 35, in dcgan.complete(args) File "/home/mohammed/dcgan-completion.tensorflow/model.py", line 220, in complete os.makedirs(os.path.join(config.outDir, 'hats_imgs'), exist_ok=True) TypeError: makedirs() got an unexpected keyword argument 'exist_ok'

    Help me out Please. Thanks

    opened by yaserali542 4
  • sorry i don't know how to start a new train

    sorry i don't know how to start a new train

    i delete the checkpoint directory and use code: ./train-dcgan.py --dataset ./data/your-dataset/aligned --epoch 20

    but everytime it tells me: "An existing model was not found in the checkpoint directory. Initializing a new one"

    but how to start to train a new one? or how to initializing a new one?

    can you help me?

    opened by lavalse 4
  • Adapt the code for custom input format (different input size / channels)

    Adapt the code for custom input format (different input size / channels)

    First of all. Thanks a lot for sharing your code!

    Do you have a suggestion which parts of the code I would need to adapt in order to make it work with 1-channel images that are larger than 64x64 pixels. I tried to just change the image_size parameter, but that produces an error, unless I set the is_crop parameter to True, but that again would reduce the input size to 64x64.

    I assume that I have to modify the get_image function?

    def imread(path,c_dim):
        if c_dim == 3:
            im  = scipy.misc.imread(path, mode='RGB').astype(np.float)
        elif c_dim == 1:
            im  = scipy.misc.imread(path, mode='L').astype(np.float)
        else:
            print('Error: c_dim must be either 3 or 1')
                
        return im
    

    but it still gives me an error:

    ValueError: Cannot feed value of shape (64, 64, 128) for Tensor u'gt_labels:0', which has shape '(?, 64, 128, 1)'

    Maybe I need to add a singleton dimension for the channels...

    opened by mgarbade 3
  • error with tf 0.12

    error with tf 0.12

    ValueError: Variable discriminator/d_bn1/discriminator_1/d_bn1/discriminator_1/d_bn1/moments/moments_1/mean/ExponentialMovingAverage/biased does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?

    opened by kursatozdalli 3
  • Cannot reproduce the demo from the blog post

    Cannot reproduce the demo from the blog post

    I ran the demo from the blog post with 3 images, but I cannot reproduce the result. Instead I get something that does not look reasonable at all: completion_newiter2500

    I even ran it for 2500 iterations. The loss does not go down, it keeps oscillating. checkpoints here: https://drive.google.com/open?id=0B86WKpvkt66BeEkxeHdmcEVOTVU

    Even a non-adaptive optimization technique should give at least reasonable results. I did not trained any model (a trained model is provided), I only ran the code:

    ./openface/util/align-dlib.py data/dcgan-completion.tensorflow/data/your-dataset/raw align innerEyesAndBottomLip data/dcgan-completion.tensorflow/data/your-dataset/aligned --size 64

    ./complete.py ./data/your-test-data/aligned/* --outDir outputImages

    cd outputImages convert -delay 10 -loop 0 completed/*.png completion.gif

    Did I miss a step in the blog post?

    I use tensorflow 0.11 on a CPU on AWS EC2 c4.4xlarge

    opened by benstaf 3
  • ValueError: setting an array element with a sequence.

    ValueError: setting an array element with a sequence.

    I am getting the following error while running train-dcgan.py

    sample_images = np.array(sample).astype(np.float32)
    ValueError: setting an array element with a sequence.
    
    opened by vinodrajendran001 2
  • Sorry But I don't know what exactly mean 'delete the checkpoint directory to train a new model'

    Sorry But I don't know what exactly mean 'delete the checkpoint directory to train a new model'

    Hello, I study Face Recognition and I'm interested in your paper.

    First of all, Thank you for sharing your code. I want to train your model with my own dataset. I find your answer about same question, But I could not understand 'delete the checkpoint directory'.

    I already try to delete checkpoint folder in dcgan-completion.tensorflow and delete the checkpoint text file or all files in the checkpoint folder. But, It doesn't work.

    Could you let me know more information to train a new dataset?

    Thank you and Hope you have a nice day. Bye

    opened by eastsky0614 1
  • Final result after 1000 epochs

    Final result after 1000 epochs

    Hi all, I ran this code for 20 epochs, but it did not show desirable results. Then I increased the number of epoch to 1000 epochs. Below image belongs to the 1000th epoch. However, final result is not as perfect as that of the author. train_999_0000. Finally, the test masked images are completed as follows:
    image image Does anybody have same experience about this code?

    opened by leilyb 2
  • Cannot feed value of shape (64, 64, 64) for Tensor 'real_images:0', which has shape '(?, 64, 64, 3)'

    Cannot feed value of shape (64, 64, 64) for Tensor 'real_images:0', which has shape '(?, 64, 64, 3)'

    [*] Reading checkpoints...

    ====== An existing model was found in the checkpoint directory. If you just cloned this repository, it's a model for faces trained on the CelebA dataset for 20 epochs. If you want to train a new model from scratch, delete the checkpoint directory or specify a different --checkpoint_dir argument.

    Traceback (most recent call last): File "train-dcgan.py", line 41, in dcgan.train(FLAGS) File "E:\Engineering\7th SEM\Final Project 2\Referal Codes\dcgan-completion.tensorflow\model.py", line 212, in train feed_dict={ self.images: batch_images, self.z: batch_z, self.is_training: True }) File "C:\Users\Dell\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\client\session.py", line 929, in run run_metadata_ptr) File "C:\Users\Dell\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\client\session.py", line 1128, in _run str(subfeed_t.get_shape()))) ValueError: Cannot feed value of shape (64, 64, 64) for Tensor 'real_images:0', which has shape '(?, 64, 64, 3)'

    opened by roshan-raj 1
  • Bug in code that computes the weighs based on the mask

    Bug in code that computes the weighs based on the mask

    Hi,

    I noticed a bug in you implementation of the function def weight_context(self, mask): Although you claim in the comment that the

    #sliding window size is 7

    you use in fact a window size of 6 as indexing an array in numpy with array[i-3:i+3] returns the values at i-3, i-2,i-1, i ,i+1, i+2 and does not include i+3. Thus, you are using a window size of 6 which is not centrally symmetric and puts different weights for pixels/holes on the right than on the left or bellow than above. So just add the +1 to all the corresponding lines...

    opened by 2006pmach 0
Owner
Brandon Amos
Brandon Amos
ICRA 2021 "Towards Precise and Efficient Image Guided Depth Completion"

PENet: Precise and Efficient Depth Completion This repo is the PyTorch implementation of our paper to appear in ICRA2021 on "Towards Precise and Effic

null 232 Dec 25, 2022
[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

Large Scale Image Completion via Co-Modulated Generative Adversarial Networks, ICLR 2021 (Spotlight) Demo | Paper [NEW!] Time to play with our interac

Shengyu Zhao 373 Jan 2, 2023
Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation (RA-L/ICRA 2020)

Aerial Depth Completion This work is described in the letter "Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation", by Lucas

ETHZ V4RL 70 Dec 22, 2022
Implements the training, testing and editing tools for "Pluralistic Image Completion"

Pluralistic Image Completion ArXiv | Project Page | Online Demo | Video(demo) This repository implements the training, testing and editing tools for "

Chuanxia Zheng 615 Dec 8, 2022
Deep Image Search is an AI-based image search engine that includes deep transfor learning features Extraction and tree-based vectorized search.

Deep Image Search - AI-Based Image Search Engine Deep Image Search is an AI-based image search engine that includes deep transfer learning features Ex

null 139 Jan 1, 2023
TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

null 2.6k Jan 4, 2023
git《Commonsense Knowledge Base Completion with Structural and Semantic Context》(AAAI 2020) GitHub: [fig1]

Commonsense Knowledge Base Completion with Structural and Semantic Context Code for the paper Commonsense Knowledge Base Completion with Structural an

AI2 96 Nov 5, 2022
Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021)

Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021) An efficient PyTorch library for Point Cloud Completion.

Microsoft 119 Jan 2, 2023
[CVPR 2021 Oral] Variational Relational Point Completion Network

VRCNet: Variational Relational Point Completion Network This repository contains the PyTorch implementation of the paper: Variational Relational Point

PL 121 Dec 12, 2022
[CVPR 2021] Unsupervised 3D Shape Completion through GAN Inversion

ShapeInversion Paper Junzhe Zhang, Xinyi Chen, Zhongang Cai, Liang Pan, Haiyu Zhao, Shuai Yi, Chai Kiat Yeo, Bo Dai, Chen Change Loy "Unsupervised 3D

null 100 Dec 22, 2022
Using pretrained language models for biomedical knowledge graph completion.

LMs for biomedical KG completion This repository contains code to run the experiments described in: Scientific Language Models for Biomedical Knowledg

Rahul Nadkarni 41 Nov 30, 2022
RGB-D Local Implicit Function for Depth Completion of Transparent Objects

RGB-D Local Implicit Function for Depth Completion of Transparent Objects [Project Page] [Paper] Overview This repository maintains the official imple

NVIDIA Research Projects 43 Dec 12, 2022
MVP Benchmark for Multi-View Partial Point Cloud Completion and Registration

MVP Benchmark: Multi-View Partial Point Clouds for Completion and Registration [NEWS] 2021-07-12 [NEW ?? ] The submission on Codalab starts! 2021-07-1

PL 93 Dec 21, 2022
TuckER: Tensor Factorization for Knowledge Graph Completion

TuckER: Tensor Factorization for Knowledge Graph Completion This codebase contains PyTorch implementation of the paper: TuckER: Tensor Factorization f

Ivana Balazevic 296 Dec 6, 2022
Code for EmBERT, a transformer model for embodied, language-guided visual task completion.

Code for EmBERT, a transformer model for embodied, language-guided visual task completion.

null 41 Jan 3, 2023
[ICCV 2021 Oral] PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers

PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers Created by Xumin Yu*, Yongming Rao*, Ziyi Wang, Zuyan Liu, Jiwen Lu, Jie Zhou

Xumin Yu 317 Dec 26, 2022
Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20. model in ONNX

ONNX msg_chn_wacv20 depth completion Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20 model in

Ibai Gorordo 19 Oct 22, 2022
PyTorch implementation for View-Guided Point Cloud Completion

PyTorch implementation for View-Guided Point Cloud Completion

null 22 Jan 4, 2023