Hi @alexgkendall ,

I followed this tutorial (http://nbviewer.ipython.org/github/BVLC/caffe/blob/master/examples/net_surgery.ipynb) and it has a nice visualization of confidence map for classification results.

Can we do the same thing with segnet? If you have time, it would be great if you can show this kind of demo.

Thank so much!

I got the following error when trying to run SegNet basic using only CPU. Basically the error says there is no field name called "dense_image_data_param". which it reads from segnet_basic_train.prototxt?

Error Output:

I1201 14:55:25.982578 4028 caffe.cpp:177] Use CPU. I1201 14:55:25.983726 4028 solver.cpp:48] Initializing solver from parameters: test_iter: 1 test_interval: 10000000 base_lr: 0.1 display: 20 max_iter: 10000 lr_policy: "step" gamma: 1 momentum: 0.9 weight_decay: 0.0005 stepsize: 10000000 snapshot: 1000 snapshot_prefix: "/home/crefeda/Desktop/SegNet/Models/Training/segnet_basic" solver_mode: CPU net: "/home/crefeda/Desktop/SegNet/Models/segnet_basic_train.prototxt" test_initialization: false I1201 14:55:25.984231 4028 solver.cpp:91] Creating training net from net file: /home/crefeda/Desktop/SegNet/Models/segnet_basic_train.prototxt [libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:26: Message type "caffe.LayerParameter" has no field named "dense_image_data_param". F1201 14:55:25.984504 4028 upgrade_proto.cpp:68] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /home/crefeda/Desktop/SegNet/Models/segnet_basic_train.prototxt *** Check failure stack trace: *** @ 0x7f9e2a9e4daa (unknown) @ 0x7f9e2a9e4ce4 (unknown) @ 0x7f9e2a9e46e6 (unknown) @ 0x7f9e2a9e7687 (unknown) @ 0x7f9e2ae712ae caffe::ReadNetParamsFromTextFileOrDie() @ 0x7f9e2ae5531b caffe::Solver<>::InitTrainNet() @ 0x7f9e2ae563fc caffe::Solver<>::Init() @ 0x7f9e2ae56709 caffe::Solver<>::Solver() @ 0x7f9e2ad5a883 caffe::Creator_SGDSolver<>() @ 0x40e35e caffe::SolverRegistry<>::CreateSolver() @ 0x4076cb train() @ 0x405651 main @ 0x7f9e29cf2ec5 (unknown) @ 0x405d61 (unknown) @ (nil) (unknown) Aborted (core dumped)

Hi @alexgkendall ,

Thank for uploading the evaluation script. However, could you please write a few lines about how to use this script with your CamVid dataset? I don't familiar with Matlab thought.

Thanks

I have my own dataset with 2 classes and a lot of pixels don't belong to either of the classes. I have assigned 0 to such pixels in the label image and 1,2 to the classes. I want to train only for labels 1,2 and not 0. Also how do I assign class weight in the softmax layer of the train.prototext file?

Hi @alexgkendall , thank you so much for your guide. I followed the instructions on http://mi.eng.cam.ac.uk/projects/segnet/tutorial.html to train the SegNet. But when I opened up a terminal and ran the commands: ./SegNet/caffe-segnet/build/tools/caffe train -gpu 0 -solver /SegNet/Models/segnet_basic_solver.prototxt An error occurred: It seems that the GPU is out of memory(batch_size = 1, so Memory required for data: 410926132). So I checked the GPU with the command: nvidia-smi Result: My GPU is GT 720 with 1G memory. Though the memory is small, it is much bigger than 245MB + 410MB(data required memory above) = 655MB.

So I would like to ask your advice for this issue :-) Thank you!

I have my own images that I wish to annotate. Which tool did you use to label your dataset?

1. During training, in 'segnet_train.prototxt', the final output is of 11 classes, but in the 'segnet_model_driving_webdemo.prototxt', the final output is 12 classes. I don't understand why is that.

2. Secondly, the labels of the images in the CamVid contain a different shade of grey for every class. Can you list the classes and the corresponding grey labels for them?

Hello, i am trying to run the provided tutorial. My graphics card doesn't have so much RAM (2GB), so i cant run the tutorial on images of size 360x480. Because of that i reshaped them to size of 120x240, changed the upsampling params in segnet_basic prototxt files, and trained the net using caffe. I picked a model from the iteration achieving around 90% accuracy, ad the same per class accuracy (most classes dont have that high accuracy but most of them are >= 70%).

After that i tried to run the two provided scripts (changed paths accordingly and in the compute_bn script i also changed in_h and in_w to 120 and 240). But when i run the second script, i get totaly random output (although i ran everything according to the tutorial).

Does this mean that the provided code only works for images of size 360x480 or am i doing something wrong?

When running the test_segmentation_camvid.py script i ran it on the training set also (so changed the data source to the train.txt file), and it gave me the same noisy output

I have followed the instructions given by the SegNet Tutorial along with the README.md to run Scripts/webcam_demo.py on a KITTI image sequence.

I made the necessary changes to Scripts/webcam_demo.py to:

• point to my caffe-segnet
• read in the image sequence files rather than webcam frames
• save the segmentation results to disk rather than displayed in a cv2.namedWindow

I also downloaded the weights and executed the script exactly as given in the readme file.

The script iterates over all images in the sequence and saves them to disk, however all segmentation results are just black images.

As a simple diagnostic, I saved the resized input image to disk, which appear normal.

Opening the black segmentation image within Matlab and inspecting the pixel channel values reveals that the values consist of zeros and ones without any discernible pattern.

Hello there!

I'm trying your (needless to say) awesome, codebase, and i wonder just 2 things :

• Why omitting the batch norm layer at the end in your models? it seems you put batch normalization before each non-linearity but not before the softmax (which is probably the most severe non linearity w.r.t vanishing gradients) ? I'm trying right now this additional batch norm layer right before the SoftMaxWithLoss on your original camvid dataset and it does not seem to worsen the training... even more, i think adding this batch normalization would help working with smaller batch sizes right?
• Is the convolution after (max)-unpooling better than using (unstrided) deconvolution? Is it because convolution is much faster ( equal num of reads but much less write cycles) & similar in the end for all the valid positions?

Thanks a lot again for your time and answers,

Etienne

Hi all, thank you very much for this great tutorial, i worked it through successfully with one exception at the end: File "./SegNet/Scripts/test_segmentation_camvid.py", line 77, in rgb_gt[:,:,1] = g_gt/255.0 ValueError: could not broadcast input array from shape (3,1,360,480) into shape (360,480)

when i comment these 3 lines out in the corresponding py-file the code runs through, but then i get a black picture :-)

I have to say i didn't train the net for the whole 10000 iterations, i stopped after 1000, just to take a first look if it works anyway. I don't think this error has to do with this, or? This seems more like an array conversion problem. Have you heard about this issue before?

One more note from my side: in your tutorial you have a little clerical error in these two lines:

python /Segnet/Scripts/compute_bn_statistics.py /SegNet/Models/segnet_train.prototxt /SegNet/Models/Training/segnet_iter_10000.caffemodel /Segnet/Models/Inference/ # compute BN statistics for SegNet python /Segnet/Scripts/compute_bn_statistics.py /SegNet/Models/segnet_basic_train.prototxt /SegNet/Models/Training/segnet_basic_iter_10000.caffemodel /Segnet/Models/Inference/ # compute BN statistics for SegNet-Basic

the marked "n"s should be written in upper case, just for information ;-)

Thank you again, i made a big progress with your tutorial for my own application.

Regards Stefan

I am using caffe in CPU mode. I have made the necessary changes in the solver file. However what are the changes that need to be made on this terminal command? /SegNet/caffe-segnet/build/tools/caffe train -gpu 0 -solver /SegNet/Models/segnet_solver.prototxt

Hello, thanks for sharing this repo. Could you please update the link so that the weights can be downloaded? It seems that the links are not working anymore.

http://mi.eng.cam.ac.uk/~agk34/resources/SegNet/segnet_weights_driving_webdemo.caffemodel

Thank you in advance

Hello,

For the multi-class segmentation, I want to know how to set different levels of the mask in y_train_dir as mask[..., 0], mask[..., 1] ... like that? Can you explain how to realize it with my own data?

Thanks, Thiem

Hello, I wonder why you used 3x3 convolution when you predicted the last output class. SegNet itself is to speed up, isn't it much faster to use 1x1 convolution? I want to know why it's composed of 3x3 convolution.

hi sir, Kindly suggest software used for labeling multi class camvid dataset and how labelled images converted to annotated masks . How we can do it for custom dataset.

Hi, I managed to train and test the Bayesian SegNet model with the default number of classes on the CamVid dataset (11). Now I'd like to train that on only 2 classes (path/background + ignore) which I managed to get working with standard SegNet by simply changing the output size.

Now when I perform training on Bayesian SegNet using CamVid, I get a solid score for each class plus low values of the loss function, so the network seems to train as it should. Now when I run the script test_bayesian_segnet.py, the entire output is saturated to class 1.0 but works fine for the original number of classes (11).

Any help is appreciated! Regards, Filip

During my testing, I found that there was always 100% for 0 class and 0% for 1 class in binary case. Actually it's not the problem of code but the label files. If the label files are uint16, the 1 pixels will be regarded as 0 so that the model can't be trained. I tried to transform them into uint8 by Matlab then it works.

Hope this helps for other guys and thanks for this outstanding code.