Thanks so much for sharing your code! I'm trying to run it from the start, but have a problem during training phase. Appreciate you support in finding a root cause.

The command I run to train U-NET, paths are adjusted for the defaults: $ th main.lua produces error log

Setting up data loader using data/train.h5  
Data loader setup done! 
...
Epoch : 1, Learning Rate : 1.00000  
THCudaCheck FAIL file=/home/david/torch/extra/cutorch/lib/THC/generic/THCTensorCopy.c line=81 error=77 : an illegal memory access was encountered
/home/david/torch/install/bin/luajit: cuda runtime error (77) : an illegal memory access was encountered at /home/david/torch/extra/cutorch/lib/THC/generic/THCStorage.c:147

Environment: Ubuntu 14.04, Titan X, CUDA 7.5, cuDNN v.5

Possible root causes:

1. I tried to temporary remove SpatialMaxPooling module, following this discussion https://groups.google.com/forum/m/#!msg/torch7/Ru-I6vP2ql0/s2vOsKoVBgAJ Finally, I simplified the NN to include no modules, but the problem persists. So, the SpatialMaxPooling is not problematic.
2. I think that the created dataset in hdf5 format has some problems. I'll try to check its correctness. If you know how to check correctness, please advice.
3. I recently switched to cuDNN v.5. Could this version be problematic?

Thanks!

danieleewww@danielUbuntu1:~/Documents/kaggleUNS/ultrasound-nerve-segmentation-using-torchnet-master$ th main.lua Setting up data loader using data/train.h5
Data loader setup done! { savePath : "data/saved_models/" valSize : 25 dataset : "train" valDataset : { __dataset : { load : function: 0x40d7dc30 list : LongTensor - size: 1198 } __replacement : false __perm : LongTensor - size: 25 __size : 25 __sampler : function: 0x407e70b0 } optimMethod : "sgd" valBatchSize : 32 cvParam : 2 maxepoch : 250 trainSize : 100 trainDataset : { __dataset : { load : function: 0x413929d0 list : LongTensor - size: 4437 } __replacement : false __perm : LongTensor - size: 100 __size : 100 __sampler : function: 0x41470da8 } trainBatchSize : 64 model : "models/unet.lua" } /home/danieleewww/torch/install/bin/luajit: ...e/danieleewww/torch/install/share/lua/5.1/trepl/init.lua:384: module 'models/unet.lua' not found:No LuaRocks module found for models/unet.lua no field package.preload['models/unet.lua'] no file '/home/danieleewww/.luarocks/share/lua/5.1/models/unet/lua.lua' no file '/home/danieleewww/.luarocks/share/lua/5.1/models/unet/lua/init.lua' no file '/home/danieleewww/torch/install/share/lua/5.1/models/unet/lua.lua' no file '/home/danieleewww/torch/install/share/lua/5.1/models/unet/lua/init.lua' no file './models/unet/lua.lua' no file '/home/danieleewww/torch/install/share/luajit-2.1.0-beta1/models/unet/lua.lua' no file '/usr/local/share/lua/5.1/models/unet/lua.lua' no file '/usr/local/share/lua/5.1/models/unet/lua/init.lua' no file '/home/danieleewww/.luarocks/lib/lua/5.1/models/unet/lua.so' no file '/home/danieleewww/torch/install/lib/lua/5.1/models/unet/lua.so' no file '/home/danieleewww/torch/install/lib/models/unet/lua.so' no file './models/unet/lua.so' no file '/usr/local/lib/lua/5.1/models/unet/lua.so' no file '/usr/local/lib/lua/5.1/loadall.so' no file '/home/danieleewww/.luarocks/lib/lua/5.1/models/unet.so' no file '/home/danieleewww/torch/install/lib/lua/5.1/models/unet.so' no file '/home/danieleewww/torch/install/lib/models/unet.so' no file './models/unet.so' no file '/usr/local/lib/lua/5.1/models/unet.so' no file '/usr/local/lib/lua/5.1/loadall.so' stack traceback: [C]: in function 'error' ...e/danieleewww/torch/install/share/lua/5.1/trepl/init.lua:384: in function 'require' ...und-nerve-segmentation-using-torchnet-master/machine.lua:52: in function 'LoadModel' ...und-nerve-segmentation-using-torchnet-master/machine.lua:34: in function '__init' ...e/danieleewww/torch/install/share/lua/5.1/torch/init.lua:91: in function <...e/danieleewww/torch/install/share/lua/5.1/torch/init.lua:87> [C]: in function 'Machine' main.lua:54: in function 'main' main.lua:59: in main chunk [C]: in function 'dofile' ...ewww/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x00406670

Any suggestion for this?

Thank you for the code. When i try to train the model with th main.lua, i get the following error at cudnn.convert in machine.lua file. Please help

attempt to call method 'replace' (a nil value) /<>/torch/install/bin/luajit: /<>/share/lua/5.1/cudnn/convert.lua:59: attempt to call method 'replace' (a nil value) stack traceback: /<>/share/lua/5.1/cudnn/convert.lua:59: in function 'convert' /<>/torchnet-master/machine.lua:66: in function 'LoadModel' /<>/torchnet-master/machine.lua:36: in function '__init' /<>/lua/5.1/torch/init.lua:91: in function </<>share/lua/5.1/torch/init.lua:87> [C]: in function 'Machine' main.lua:54: in function 'main' main.lua:59: in main chunk [C]: in function 'dofile' ...plab/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk

[C]: at 0x004065d0

I' new to lua and torch. I change default value of trainSize/valSize to -1 to use whole dataset. I run this scripts with epoch around 200, I used each 50/100/150/200 epoch checkpoint to generate submission but only get LB ~0.56. Am I doing wrong ?

Thank you for sharing the code! I am running into a problem with the main.lua. I am not able to require the unet.lua provided with the install.

hokie23@ubuntu03:/ubuntu/scratch/hokie23/noirb/GRDT2/source$ th main.lua Setting up data loader using /ubuntu/scratch/hokie23/noirb/GRDT2/train/data.h5 Data loader setup done! { savePath : "data/saved_models/" valSize : 25 dataset : "data" valDataset : { __dataset : { load : function: 0x41e4ec40 list : LongTensor - size: 1198 } __replacement : false __perm : LongTensor - size: 25 __size : 25 __sampler : function: 0x41c01f48 } optimMethod : "sgd" valBatchSize : 32 cvParam : 2 maxepoch : 250 trainSize : 100 trainDataset : { __dataset : { load : function: 0x412fa9c8 list : LongTensor - size: 4437 } __replacement : false __perm : LongTensor - size: 100 __size : 100 __sampler : function: 0x41bdadd8 } trainBatchSize : 64 model : "models/unet.lua" } /home/hokie23/torch/install/bin/luajit: .../hokie23/torch/install/share/lua/5.1/trepl/init.lua:384: module 'models/unet.lua' not found:No LuaRocks module found for models/unet.lua no field package.preload['models/unet.lua'] no file '/home/hokie23/.luarocks/share/lua/5.1/models/unet/lua.lua' no file '/home/hokie23/.luarocks/share/lua/5.1/models/unet/lua/init.lua' no file '/home/hokie23/torch/install/share/lua/5.1/models/unet/lua.lua' no file '/home/hokie23/torch/install/share/lua/5.1/models/unet/lua/init.lua' no file './models/unet/lua.lua' no file '/home/hokie23/torch/install/share/luajit-2.1.0-alpha/models/unet/lua.lua' no file '/usr/local/share/lua/5.1/models/unet/lua.lua' no file '/usr/local/share/lua/5.1/models/unet/lua/init.lua' no file '/home/hokie23/.luarocks/lib/lua/5.1/models/unet/lua.so' no file '/home/hokie23/torch/install/lib/lua/5.1/models/unet/lua.so' no file '/home/hokie23/torch/install/lib/models/unet/lua.so' no file './models/unet/lua.so' no file '/usr/local/lib/lua/5.1/models/unet/lua.so' no file '/usr/local/lib/lua/5.1/loadall.so' no file '/home/hokie23/.luarocks/lib/lua/5.1/models/unet.so' no file '/home/hokie23/torch/install/lib/lua/5.1/models/unet.so' no file '/home/hokie23/torch/install/lib/models/unet.so' no file './models/unet.so' no file '/usr/local/lib/lua/5.1/models/unet.so' no file '/usr/local/lib/lua/5.1/loadall.so' stack traceback: [C]: in function 'error' .../hokie23/torch/install/share/lua/5.1/trepl/init.lua:384: in function 'require' ...scratch/hokie23/noirb/GRDT2/source/machine.lua:52: in function 'LoadModel' ...scratch/hokie23/noirb/GRDT2/source/machine.lua:34: in function '__init' .../hokie23/torch/install/share/lua/5.1/torch/init.lua:91: in function <.../hokie23/torch/install/share/lua/5.1/torch/init.lua:87> [C]: in function 'Machine' main.lua:54: in function 'main' main.lua:59: in main chunk [C]: in function 'dofile' ...alli/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x00406670

The kaggle link for downloading the dataset is not working. Can you please provide alternate cloud / drive link to access and download the data? Please Help!

this issue was fixed by create a saved_models directory under /data. Otherwise, the execution of main.lua will render the following error in which the training model could not be saved at all.

danieleewww@daniel-System-UbuntuGT950:~/torch/ultrasound-nerve-segmentation-using-torchnet$ th main.lua Setting up data loader using ./data/train.h5
Data loader setup done! { optimMethod : "sgd" trainBatchSize : 32 dataset : "train" trainDataset : { __replacement : false __sampler : function: 0x10660f30 __perm : LongTensor - size: 100 __size : 100 __dataset : { load : function: 0x1082de40 list : LongTensor - size: 4437 } } savePath : "data/saved_models/" trainSize : 100 valDataset : { __replacement : false __sampler : function: 0x107daee0 __perm : LongTensor - size: 25 __size : 25 __dataset : { load : function: 0x10661640 list : LongTensor - size: 1198 } } valSize : 25 valBatchSize : 32 cvParam : 2 model : "./models/unet.lua" maxepoch : 250 } Epoch : 1, Learning Rate : 1.00000
[==================== 4/4 ====================>] Tot: 3s873ms | Step: 1s291ms /home/danieleewww/torch/install/bin/lua: cannot open </home/danieleewww/torch/ultrasound-nerve-segmentation-using-torchnet/data/saved_models/unet_512_train_sgd_epoch_torchnet_1.t7> in mode w at /home/danieleewww/torch/pkg/torch/lib/TH/THDiskFile.c:649 stack traceback: [C]: in ? [C]: in function 'DiskFile' ...e/danieleewww/torch/install/share/lua/5.2/torch/File.lua:385: in function 'save' ./machine.lua:132: in function 'saveModels' ./machine.lua:197: in function <./machine.lua:193> (...tail calls...) ...ch/install/share/lua/5.2/torchnet/engine/optimengine.lua:122: in function <...ch/install/share/lua/5.2/torchnet/engine/optimengine.lua:67> (...tail calls...) ./machine.lua:110: in function 'train' main.lua:55: in function 'main' main.lua:59: in main chunk [C]: in function 'dofile' ...ewww/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: in ? Time : 8.32288 | Training Dice Score : 0.04045 | Norm Ratio : 0.10308 | GradParam Norm : 9.67324 | Validation Loss : 0.02785 | Param Norm : 93.83880 | Validation Dice Score : 0.52000 | Training Loss : 0.08134 |

using "th create_dataset.lua -train /Users/mac/USNerve/train/images -trainOutput /Users/j-titus/USNerve/train/images.h5 -test /Users/j-titus/USNerve/test -testOutput /Users/j-titus/USNerve/test.h5"

throws the error /Users/j-titus/torch/install/bin/luajit: create_dataset.lua:82: table index is nil stack traceback: create_dataset.lua:82: in function 'create_train_h5' create_dataset.lua:118: in main chunk [C]: in function 'dofile' .../j-titus/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk [C]: at 0x010f836a10

is it a path error or have I missed something? All of my train and mask png files are in the same folder