Keras-tensorflow implementation of Fully Convolutional Networks for Semantic Segmentation（Unfinished）

train.py is unable to train the model, i changed accordingly but then i getting the .error

---> File "C:\Users\Prisilla\Desktop\Semantics\keras_FCN\xyz\SegDataGenerator.py", line 174, in init fp = open(file_path)

---> FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\Prisilla/.keras/VOCdevkit/VOC2012/ImageSets/Segmentation/train.txt' --> unable to create checkpoint_weights.hdf5

inference.py file is showing error File "C:/Users/Prisilla/Desktop/Semantics/keras_FCN/Sinference.py", line 92, in results = Sinference(model_name, weight_file, image_size, image_list, data_dir, label_dir) File "C:/Users/Prisilla/Desktop/Semantics/keras_FCN/Sinference.py", line 35, in Sinference model.load_weights(checkpoint_path, by_name=True) OSError: Unable to open file (unable to open file: name = 'C:\Users\Prisilla\Desktop\Semantics\keras_FCN\Models/DenseNet_FCN\checkpoint_weights.hdf5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

Can any one tell what changes can be done to run the model. I am trying to learn and work on semantic segmentation... but the code looks like not executing

I train the model on VOC2012 segmentation, using 250 epoch and 20 batch_size. The training accu is 0.985~.(It should be the pixel accu, right?) But the testing mIoU is just 0.56025, which has a big gap with the result in the paper. What's your result? And maybe the hyperparameter is not optimal?

[ 0.9077976 0.75288631 0.47242709 0.58043887 0.50712961 0.47588975
  0.72664924 0.68814923 0.66110743 0.20972129 0.53752346 0.31540707
  0.5731418 0.50105971 0.64968748 0.76978092 0.43357291 0.5765887
  0.32090376 0.61638412 0.49687383]
meanIOU: 0.560625
pixel acc: 0.898940

Hello, thank you for your code. When I try to run the train.py to do a demo, but I get a "TypeError: pad_width must be of integral type.", and it points to the "Keras-FCN\utils\SegDataGenerator.py", line 238, in next x = np.lib.pad(x, ((pad_h / 2, pad_h - pad_h / 2), (pad_w / 2, pad_w - pad_w / 2), (0, 0)), 'constant', constant_values=0.)". I don't know how to fix this problem, can you give me some suggestions?

Hello,

Thank you for providing your code. I have been able to successfully training and evaluate data using the VOC2011 and VOC 2012 datasets with the model "AtrousFCN_Resnet50_16s".

I am working on training using my own data and the model "AtrousFCN_Resnet50_16s". I have a series of rgb .jpgs which I have segmented similarly to the VOC2011 and VOC2012 datasets. I am training for 3 classes (G1, G2, and no data) which are represented by 3 colors (purple, green, black), the first of two do not match the colors of the VOC models and are new categories. I should add that following the VOC data, I've also included a VOID category that is an outline around my objects of interest.

I have generated code to match the format of the VOC2012 dataset call in train.py. In this duplication, I have also changed the number of classes to 3:

if dataset == 'Grapes': train_file_path = os.path.expanduser('~/path/to/my/data/train.txt') val_file_path = os.path.expanduser('~/path/to/my/data/val.txt') data_dir = os.path.expanduser('~/path/to/my/data/JPEGImages') label_dir = os.path.expanduser('~/path/to/my/data/SegmentationObject') data_suffix='.jpg' label_suffix='.jpg' classes = 3

When I run my code, I get an error in the SegDataGenerator which reads: ValueError: could not broadcast input array from shape (350,350,3) into shape (350,350,1)

I've tracked back and the error occurs here in the code:

        if self.loss_shape is not None:
            y = np.reshape(y, self.loss_shape)

        batch_x[i] = x
        batch_y[i] = y

I noticed that in your COCO implementation, you added loss_shape and I added "loss_shape = (target_size[0] * target_size[1] * classes,)" to the configuration, but now I get a new error:

ValueError: Cannot feed value of shape (1, 367500) for Tensor 'bilinear_up_sampling2d_1_target:0', which has shape '(?, ?, ?, ?)'

Do you have any suggestions for what I need to do so that I can train on my own data? I might add that my input data has fewer rows and columns than the VOC data by about 50%.

Thank you for your response.

HI, I'm new in this field, I use your code to do segmentation. I confuse about the loss func. For example I have two classes. [0] represents background class and [1] represent another class. My label '.png' has the shape like [0 0 0 0 1 1 0 0...] (2dim). Now I use softmax_sparse_crossentropy_ignoring_last_label(y_true, y_pred) as my loss function.

It says

ignoring_last_label

I'm wondering which label it will ignore? class[0], class[1], or..another? I trace the code and find ..it seems still have 2 classes shape.

my y_pred, the out put of my model has the shape (num,224,224,2) And I want to calculate 2 classes loss.

Many Thanks!

When I run python train.py, it goes wrong. I wander where is the h5 format file. Could you please help me with this problem

Traceback (most recent call last):
  File "train.py", line 120, in <module>
    data_dir, label_dir, target_size=target_size, batchnorm_momentum=batchnorm_momentum, resume_training=resume_training)
  File "train.py", line 49, in train
    model = globals()[model_name](weight_decay=weight_decay, input_shape=input_shape, batch_momentum=batchnorm_momentum)
  File "/home/cv/keras-fcn/Keras-FCN-master/models.py", line 209, in AtrousFCN_Resnet50_16s
    model.load_weights(weights_path, by_name=True)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 2702, in load_weights
    f = h5py.File(filepath, mode='r')
  File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/files.py", line 272, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
  File "/usr/local/lib/python2.7/dist-packages/h5py/_hl/files.py", line 92, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2684)
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip-4rPeHA-build/h5py/_objects.c:2642)
  File "h5py/h5f.pyx", line 76, in h5py.h5f.open (/tmp/pip-4rPeHA-build/h5py/h5f.c:1930)
IOError: Unable to open file (Unable to open file: name = '/home/cv/.keras/models/fcn_resnet50_weights_tf_dim_ordering_tf_kernels.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0)

Hello,

I have successfully trained the AtrousFCN_Resnet50_16s model with my own data and am having difficulty getting the inference.py module to run. I am hoping that you might have some insight for how I can get the model up and running. Can you help? Thanks! See below:

First a little about the model: My own training data has 3 values: 0 corresponds to the VOID layer in the VOC Data and is the outline of my objects. 1 corresponds to the segmentation pixels I am looking for. And 2 corresponds to background. When I trained the model, I set the number of classes to 2 with the desire/expectation that the model would train to identify categories 1 and 2 in the same fashion as the VOC2011 data. I used the loss function "softmax_sparse_crossentropy_ignoring_last_label" that was included with your code. Here are my results after 250 epochs. From the shapes of the plots, I believe my model is training correctly. As an output, in the AtrousFCN_Resnet50_16s folder, I get a new file with the timestamp for model completion titled 'checkpoint_weights.hdf5' which I understand as the new weights/biases for the trained model.

Now, I am working on now evaluating the model running evaluation.py using the AtrousFCN_Resnet50_16s model and the new checkpoint file. I have set nb_classes to 2, and pointed the code to use the new checkpoint file for weights. When I run the code I receive the following error:

ValueError: Dimension 0 in both shapes must be equal, but are 1 and 2 for 'Assign_318' (op: 'Assign') with input shapes: [1,1,2048,21], [2,2048,1,1].

This error traces back to line 32 of inference.py: model.load_weights(checkpoint_path, by_name=True)

Interestingly, when I instead point to the weight file I used when I initialized training ('fcn_resnet50_weights_tf_dim_ordering_tf_kernels.h5), I do not get this error. I think the problem is with inference.py interpreting the new model weights file or with the model itself.

Will you verify that I am pointing to the correct model weights file for evaluation?

I also see that line 31 in inference.py in my code reads: `model = globals()[model_name](batch_shape=batch_shape, input_shape=(512, 512, 3))', and line 21 of my code reads: 'batch_shape = (1, ) + image_size + (3,)'.

When I look at my error again (> ValueError: Dimension 0 in both shapes must be equal, but are 1 and 2 for 'Assign_318' (op: 'Assign') with input shapes: [1,1,2048,21], [2,2048,1,1].) I notice that the matrices look reversed, and that the model is looking for 21 classes instead of 2 (last digit versus first digit). What do I need to adjust in my code to fix this error? Do I need to change the batch shape? Is there a spot during model initialization where I am incorrectly loading weights? Do I need to adjust something in the SegDataGenerator to get it to work? Thank you for your input.

Hello aurora95

I am trying to use SegDataGenerator. However, it is too slow. I am running the code on TitanX and 64 GB memory. the output of my network is (None, 50176, 2). Therefore, I have modified SegDataGenerator by making the shape of y equals to (None, 50176, 2 ). In addition, I am using categorical cross entropy. batch_size =64 My question is: why loading the training images is very slow? For example, one epoch needs 151051s.

Thank you

Is the .h5 weights file downloaded from somewhere else? Or did you train your weights?

For example in model.py

weights_path = os.path.expanduser(os.path.join('~', '.keras/models/fcn_vgg16_weights_tf_dim_ordering_tf_kernels.h5'))
model.load_weights(weights_path, by_name=True)
return model

Based on the file path .keras/, it looks like a weights file come from Keras. But I am not aware of that keras provides fcn weights.

When I run "git clone [email protected]:ahundt/keras-contrib.git -b densenet-atrous" there is a mistake :

Permission denied (publickey). fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.

So I find another keras- contrib ： git clone https://www.github.com/farizrahman4u/keras-contrib.git

however when i run train.py there is a mistake: Traceback (most recent call last): File "train.py", line 15, in from models import * File "/home/deeplearning/src/Keras-FCN/models.py", line 6, in from keras_contrib.applications import densenet File "build/bdist.linux-x86_64/egg/keras_contrib/init.py", line 13, in File "build/bdist.linux-x86_64/egg/keras_contrib/optimizers/init.py", line 1, in File "build/bdist.linux-x86_64/egg/keras_contrib/optimizers/ftml.py", line 8, in File "build/bdist.linux-x86_64/egg/keras_contrib/optimizers/ftml.py", line 34, in FTML AttributeError: 'module' object has no attribute 'legacy_get_updates_support'

I think there's a bug in the cropping functions. In particular, I believe the correct order should be (h, w) instead of (w, h).

In the constructor, self.row_index (height) is set to 1 and self.col_index (width) to 2. However, in the crop functions, e.g. pair_random_crop, the width dimension is cropped as the first dimension.

Maybe I'm missing something?

Hello, I am trying to understand the project but I can´t. The weights that you can download, which models are they from, as there are several models, and which dataset are they trained on? Thanks

I am a beginner, and I do exactly as Readme.md tell me in Git bash using Windows7, when I run this command python data_pascal_voc.py pascal_voc_setup I got this error as follows.

Using TensorFlow backend. Traceback (most recent call last): File "data_pascal_voc.py", line 28, in from tf_image_segmentation.utils.tf_records import write_image_annotation_pairs_to_tfrecord ModuleNotFoundError: No module named 'tf_image_segmentation'

Please help me how to install 'tf_image_segmentation'! Thank you so much!

Hi,

Thanks a lot for the implementation!

Regarding upsampling, you used a custom BilinearUpSampling2D layer, which I guess cannot be learned (sorry if I'm mistaken).

I believe a better solution is to use the built-in Conv2DTranspose Keras layer for upconvolution (c.f. section 3.3 of the FCN paper).

Best regards.

Hello, I also have this problem, I want to train my data set, but the picture of my data set is jpg, the label is png, can I use this data set for training, if I can, I need the program What changes have been made, thank you

UnboundLocalError Traceback (most recent call last) in () 1 train(batch_size, epochs, lr_base, lr_power, weight_decay, classes, model_name, train_file_path, val_file_path, 2 data_dir, label_dir, target_size=target_size, batchnorm_momentum=batchnorm_momentum, resume_training=resume_training, ----> 3 class_weight=class_weight, loss_fn=loss_fn, metrics=metrics, loss_shape=loss_shape, ignore_label=ignore_label, label_cval=label_cval)

in train(batch_size, epochs, lr_base, lr_power, weight_decay, classes, model_name, train_file_path, val_file_path, data_dir, label_dir, target_size, batchnorm_momentum, resume_training, class_weight, dataset, loss_fn, metrics, loss_shape, label_suffix, data_suffix, ignore_label, label_cval) 140 # ), 141 # nb_val_samples = 64 --> 142 class_weight=class_weight 143 ) 144

c:\users\kkq\appdata\local\programs\python\python36\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs) 89 warnings.warn('Update your ' + object_name + 90 ' call to the Keras 2 API: ' + signature, stacklevel=2) ---> 91 return func(*args, **kwargs) 92 wrapper._original_function = func 93 return wrapper

c:\users\kkq\appdata\local\programs\python\python36\lib\site-packages\keras\engine\training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch) 1413 use_multiprocessing=use_multiprocessing, 1414 shuffle=shuffle, -> 1415 initial_epoch=initial_epoch) 1416 1417 @interfaces.legacy_generator_methods_support

c:\users\kkq\appdata\local\programs\python\python36\lib\site-packages\keras\engine\training_generator.py in fit_generator(model, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch) 171 for m in model.stateful_metric_functions: 172 m.reset_states() --> 173 callbacks.on_epoch_begin(epoch) 174 steps_done = 0 175 batch_index = 0

c:\users\kkq\appdata\local\programs\python\python36\lib\site-packages\keras\callbacks.py in on_epoch_begin(self, epoch, logs) 61 logs = logs or {} 62 for callback in self.callbacks: ---> 63 callback.on_epoch_begin(epoch, logs) 64 self._delta_t_batch = 0. 65 self._delta_ts_batch_begin = deque([], maxlen=self.queue_length)

c:\users\kkq\appdata\local\programs\python\python36\lib\site-packages\keras\callbacks.py in on_epoch_begin(self, epoch, logs) 627 lr = float(K.get_value(self.model.optimizer.lr)) 628 try: # new API --> 629 lr = self.schedule(epoch, lr) 630 except TypeError: # old API for backward compatibility 631 lr = self.schedule(epoch)

in lr_scheduler(epoch, mode) 49 lr = 0.1 50 ---> 51 print('lr: %f' % lr) 52 return lr 53 scheduler = LearningRateScheduler(lr_scheduler)

UnboundLocalError: local variable 'lr' referenced before assignment