I use GTX 1080 (8G). The size of images in train datasets are 640*480. When the batch size is larger than 5, GPU will out of memory. The weight is 515MB, so is there something wrong? Second, the training is very slow. If batch size is 2, five seconds are needed. Do you know the reason, thanks!@MarvinTeichmann

@MarvinTeichmann thank you for this implementation of FCN in tensorflow. This is exactly what I was looking for. I'd like to humbly suggest you port your work into tflearn. I think that your implementation of _upscore_layer as well as the loss function you've defined would be very useful in tflearn.

Hello Marvin,

I have some trouble when I run your codes.

When I run the test_fcn32_vgg.py, it reminds me that there is no enough memory to allocate the tensor. (with gpu) However, I can run the test_fcn16_vgg.py(with gpu), it works well. And then I tried running with only cpu, both test_fcn32_vgg.py and test_fcn16_vgg.py works well.

I read the codes and it seems that test_fcn16_vgg.py may need more memory?

Is it just for that my memory of GPu is not enough (it is about 2GB) or something else?

I run the test code, but the result is not so good.

And you can see the part of the dog is missing, which is not like your result on github.

Thanks for your time.

Sincerely, Shuhong

Hi, all,

I got the following error when running "python test_fcn32_vgg.py"

InternalError (see above for traceback): Dst tensor is not initialized.
	 [[Node: fc6/weights/Initializer/Const = Const[_class=["loc:@fc6/weights"], dtype=DT_FLOAT, value=Tensor<type: float shape: [7,7,512,4096] values: [[[1.9745843e-05 0.00035308721 -0.0018327669]]]...>, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

My version of tf is r0.12.1 and I checkout the package (bf9400c6303826e1c25bf09a3b032e51cef57e3b) already. Could someone suggest me how to fix it?

THX!

Hi! I got this error.

npy file loaded Layer name: conv1_1 Layer shape: (3, 3, 3, 64) Traceback (most recent call last): File "test_fcn32_vgg.py", line 24, in vgg_fcn.build(batch_images, debug=True) File "/home/sheldon/tensorflow-fcn-master/fcn32_vgg.py", line 74, in build self.conv1_1 = self._conv_layer(bgr, "conv1_1") File "/home/sheldon/tensorflow-fcn-master/fcn32_vgg.py", line 135, in _conv_layer filt = self.get_conv_filter(name) File "/home/sheldon/tensorflow-fcn-master/fcn32_vgg.py", line 259, in get_conv_filter tf.add_to_collection(tf.GraphKeys.REGULARIZATION_LOSSES, AttributeError: type object 'GraphKeys' has no attribute 'REGULARIZATION_LOSSES'

python 2.7 tensorflow1.2 Could you give me some suggestions?

Hi, I am fine-tuning your FCN toward my own dataset. But after tuning, I was doing stuff like 'conv1_1/filter:0','conv1_1/biases:0'; sess.run(filters) to extract features and save them into npy file. But since we have lots of filters here, listing all of them would be messy.

So I was wondering if we can do it with the tf.train.Saver, which allows us to store the trained Variables fast and clean. But later I notice that there is no "Variable" in the fcn8s_vgg file. And I am thinking that by using your original fcn8s_vgg.py file, we can't use the tf.train.Saver(), right?

I am new to tensorflow, please let me know your comments, thanks.

   epsilon = tf.constant(value=1e-4)     
   logits = logits + epsilon
   softmax = tf.nn.softmax(logits)

It should be epsilon = tf.constant(value=1e-4)
softmax = tf.nn.softmax(logits) + epsilon

After reading the readme file, it's still not clear to me what data set was used for training? I understand some filters are initialized from vgg16, then training is done on what dataset?

By "No Pascal VOC finetuning was applied to the weights.", I guess the training set is not Pascal VOC?

Thanks,

Hi Mr.Marvin Teichmann I found this in the usage that: The images is a tensor with shape [None, h, w, 3]. Where h and w can have arbitrary size.

My training data is gray images with no rgb channels. Can I still use this code and vgg to train them?

Hi Marvin,

Thank you for sharing the code. May I ask you for some advice?

I am trying to adopting your code for one of my data set, which has about 300 images with 600*800 sizes. The masked one has about two classes, class 1 takes about 20% of the whole image. The image itself is kind of far away from the data set used to pre-train the VGG model. For instance, the bio-medical data.

There are several questions,

1. Can I still use the pre-trained VGG weights?
2. Which part of the code need I change to incorporate the two-classes case?
3. Since the image size is pretty big and the number of images is limited, I have been planning to conduct patch-wise training, i.e., creating sample patches from the original images. Should I heavily sample the areas corresponding to class 1? During the training process, do I have to put all the patches corresponding to a single image into a single batch? Or I can just use batch size=1 in the training process?

Wenouyang

I found code about the input shape.

    if target_size:
        input_shape = target_size + (3,)
    else:
        input_shape = (None, None, 3)

When I set target_size=None, error encountered as below:

Traceback (most recent call last):
  File "train.py", line 132, in <module>
    data_dir, label_dir, target_size=target_size, resume_training=True)
  File "train.py", line 102, in train
    batch_size=batch_size, shuffle=True
  File "/data/kuixu/exper/fcn/Keras-FCN/utils/SegDataGenerator.py", line 186, in flow_from_directory
    save_to_dir=save_to_dir, save_prefix=save_prefix, save_format=save_format)
  File "/data/kuixu/exper/fcn/Keras-FCN/utils/SegDataGenerator.py", line 26, in __init__
    self.target_size = tuple(target_size)
TypeError: 'NoneType' object is not iterable

please help

Hello! I found that in fcn8s_vgg.py. The weight decay loss is not added into tf.GraphKeys.REGULARIZATION_LOSSES, since you specify another collection name.

I would like to know if this will affect the final results?

when I convert the ckpt model to movidius graph, I meet the problem: InvalidArgumentError (see above for traceback): Number of ways to split should evenly divide the split dimension, but got split_dim 3 (size = 224) and num_split 3 [[Node: Validation/Validation/Processing/split = Split[T=DT_FLOAT, num_split=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Validation/Validation/Processing/split/split_dim, Validation/ExpandDims)]]」

then I find the code in tensorflow split_op.cc OP_REQUIRES(context, input_shape.dim_size(split_dim) % num_split == 0, errors::InvalidArgument( "Number of ways to split should evenly divide the split " "dimension, but got split_dim ", split_dim, " (size = ", input_shape.dim_size(split_dim), ") ", "and num_split ", num_split));

I read code in KittiSeg/submodules/tensorflow-fcn/fcn8_vgg.py , the following code execute "tf.split()" function.

         red, green, blue = tf.split(rgb, 3, 3)

        # assert red.get_shape().as_list()[1:] == [224, 224, 1]
        # assert green.get_shape().as_list()[1:] == [224, 224, 1]
        # assert blue.get_shape().as_list()[1:] == [224, 224, 1]
        bgr = tf.concat([
            blue - VGG_MEAN[0],
            green - VGG_MEAN[1],
            red - VGG_MEAN[2],
        ], 3)

this code means the image channels which divided by 3, but the log shows 224 divided 3 , so 224%3 != 0.

I can't understand what codes below want to do, does anybody can explain? Thanks so much !!!

if use_dilated:
    pad = [[0, 0], [0, 0]]
    self.pool4 = tf.nn.max_pool(self.conv4_3, ksize=[1, 2, 2, 1],
                                strides=[1, 1, 1, 1],
                                padding='SAME', name='pool4')
    self.pool4 = tf.space_to_batch(self.pool4,
                                   paddings=pad, block_size=2)
else:
    self.pool4 = self._max_pool(self.conv4_3, 'pool4', debug)

After loading the weights still the output is random. What should i change in the code after giving correct path of .npy file ?? Please tell me how to predict loading correct weights my output image :-

Hi Marvin,thanks for your code! I have two problems. Can you help me ? First,I want to know why you convert RGB to BGR in fcn8_vgg ? second,what's the meaning of the code as follows ( in line 123) : if use_dilated: self.pool5 = tf.batch_to_space(self.pool5, crops=pad, block_size=2) self.pool5 = tf.batch_to_space(self.pool5, crops=pad, block_size=2) self.fc7 = tf.batch_to_space(self.fc7, crops=pad, block_size=2) self.fc7 = tf.batch_to_space(self.fc7, crops=pad, block_size=2)