When I try test2d.py, the error occured: python3 test2d.py --task fundus --split all --ds valid2 --net segtran --bb resnet101 --translayers 3 --layercompress 1,1,2,2 --cpdir ../model/segtran-fundus-train,valid,test,drishti,rim-05011826 --iters 9500 --outorigsize 'fundus' mean/std loaded from 'fundus-cropped-gray0.5-stats.json' 'all' 400 samples of size 576 chosen (total 800) in '../data/fundus/valid2' 'args' orig in-feat: 2048, in-feat: 2048, out-feat: 512, in-scheme: AN, out-scheme: AN, translayer_dims: [2048, 2048, 1024, 512] Namespace(ablate_multihead=False, attn_clip=500, backbone_type='resnet101', batch_size=8, bb_feat_upsize=True, binarize=False, calc_flop=False, checkpoint_dir='../model/segtran-fundus-train,valid,test,drishti,rim-05011826', debug=False, device='cuda', do_remove_frag=False, ds_class='SegCrop', ds_name='valid2', ds_split='all', eval_robustness=False, gpu='0', gray_alpha=0.5, has_FFN_in_squeeze=False, in_fpn_layers='34', in_fpn_scheme='AN', in_fpn_use_bn=False, iters='9500', job_name='fundus-valid2', mean=[0.578, 0.429, 0.318], mid_type='shared', mince_channel_props=None, mince_scales=None, net='segtran', num_attractors=256, num_classes=3, num_modalities=0, num_modes=4, num_translayers=3, num_workers=4, orig_input_size=(576, 576), out_fpn_layers='1234', out_fpn_scheme='AN', out_origsize=True, output_upscale=2.0, patch_size=(288, 288), polyformer_mode=None, pos_bias_radius=7, pos_code_type='lsinu', pos_code_weight=1.0, qk_have_bias=True, reload_mask=False, reshape_mask_type=None, robust_aug_degrees=[0.5, 1.5], robust_aug_types=None, robust_ref_cp_path=None, robust_sample_num=120, robustness_augs=None, sample_num=-1, save_ext='png', save_features_img_count=0, save_results=True, std=[0.184, 0.162, 0.144], task_name='fundus', test_interp=None, tie_qk_scheme='none', trans_output_type='private', translayer_compress_ratios=[1.0, 1.0, 2.0, 2.0], use_exclusive_masks=False, use_global_bias=False, use_mince_transformer=False, use_pretrained=True, use_squeezed_transformer=True, verbose_output=False, vis_layers=None, vis_mode=None) Segtran Fusion Encoder with 3 trans-layers Learnable Sinusoidal positional encoding Fusion0-in-squeeze: v_has_bias: False, has_FFN: False, has_input_skip: False Fusion0-in-squeeze in_feat_dim: 2048, feat_dim: 2048, qk_have_bias: True Fusion0-squeeze-out: v_has_bias: False, has_FFN: True, has_input_skip: False Fusion0-squeeze-out in_feat_dim: 2048, feat_dim: 2048, qk_have_bias: True Fusion1-in-squeeze: v_has_bias: False, has_FFN: False, has_input_skip: False Fusion1-in-squeeze in_feat_dim: 2048, feat_dim: 2048, qk_have_bias: True Fusion1-squeeze-out: v_has_bias: False, has_FFN: True, has_input_skip: False Fusion1-squeeze-out in_feat_dim: 2048, feat_dim: 1024, qk_have_bias: True Fusion2-in-squeeze: v_has_bias: False, has_FFN: False, has_input_skip: False Fusion2-in-squeeze in_feat_dim: 1024, feat_dim: 1024, qk_have_bias: True Fusion2-squeeze-out: v_has_bias: False, has_FFN: True, has_input_skip: False Fusion2-squeeze-out in_feat_dim: 1024, feat_dim: 512, qk_have_bias: True Downloading: "https://download.pytorch.org/models/resnet101-5d3b4d8f.pth" to /root/.cache/torch/hub/checkpoints/resnet101-5d3b4d8f.pth **resnet101 created** Parameter Count: 172737073 **args[backbone_type]=resnet101, checkpoint args[backbone_type]=eff-b4, inconsistent!** I find in test2d：“parser.add_argument('--bb', dest='backbone_type', type=str, default='eff-b4', help='Segtran backbone'” then find in resnet.py，when test2d.py is running，it downloads from the web101. But it has this error, I don't know how to solve this,Could you tell me how to solve this problem? thank u!

Dear Shaohua while checking the train process I see the dice is around 9% after percent of total epochs. the total training on 8 rtx 8000 nvidia takes around 25 hours estimated. what is the cause of such low value for dice? waiting to hear from you. thanks

Hi, thanks for your excellent work!

I want to re-implement this model, will you provide your trained models whose results achieve SOTA from your paper? Thus we can only use your model doing inference.

Especially model on REFUGE and BraTS, segtran is very extraordinary.

Best,

Hi dear Askerlee, first of all, greetin for your valuable code, please explain how I can test this code over brats 2020 dataset, as I replace that dataset in the brats path also setting the parameters in the train3d.py, but the .h5 files were not exist in the cases pathes. waiting to hear from you, thanks

Hi Dr. Lee, do you have the code that you used to compare with various baselines in section 5.2 (list of other models) for 2D dataset? Is there a similar comparison for the 3D dataset?

Hi，sir. I move the project to the new computer. I meet ModuleNotFoundError: No module named 'networks.segtran2d'.It would have been nice if you had given me a hint. Thank u!

Hi Dear Shaohua

is there any part of this code to set just trained model path and new sample pathe from out file.oy then getting result by these parameters without going into args parsing?

Hi Dr Lee,

I got this error when I tried to run the test command using your recently updated Brats iter_8000 checkpoint, any advice?

python3 test3d.py --task brats --split all --bs 5 --ds 2019valid --net segtran --attractors 1024 --translayers 1 --cpdir ./ --iters 8000

Traceback (most recent call last): File "test3d.py", line 426, in allcls_avg_metric = test_calculate_metric(iter_nums) File "test3d.py", line 350, in test_calculate_metric load_model(net, args, checkpoint_path) File "test3d.py", line 311, in load_model if (k not in ignored_keys) and (args2.dict[k] != cp_args[k]): KeyError: 'qk_have_bias'

The Error is on "load_model(net, args, checkpoint_path)" in the following:

for iter_num in iter_nums:
    if args.checkpoint_dir:
        checkpoint_path = os.path.join(args.checkpoint_dir, 'iter_' + str(iter_num) + '.pth')
        load_model(net, args, checkpoint_path)

Hi Dr Lee,

I tried to run test3d.py for a set of 110 test images a few times, but it somehow always exit unexpectedly around 70-ish images without any error. So far, I have never successfully finished the 110 images.

Any advice?

Best, Wendy

Hello, I am trying to reproduce the model training with 2019 Brats datasets (LGG and HGG). I am using BS = 1 due to memory limitation.

I trained 10k+ iterations now, the loss functions look like this, is this trending in the right direction or expected?

Dear sir, we want to use the code of nnU-Net on fundus (2d images). We have some problems, could you give us some guidance, thank u! We also notice you have mentioned' It is primarily designed for 3D tasks, but can also handle 2D images after converting them to pseudo-3D.' Here are our settings and traceback: The parameter are --nproc_per_node=1 --master_port=7152 /data/sementation/code/train2d.py --task fundus --ds train --split train --translayers 3 --layercompress 1,1,2,2 --net nnunet --bb resnet101 --maxiter 5000 --bs 32 --noqkbias

Traceback (most recent call last): File "<input>", line 1, in <module> File "/root/.pycharm_helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile pydev_imports.execfile(filename, global_vars, local_vars) # execute the script File "/root/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "/data/sementation/code/train2d.py", line 1434, in <module> outputs = net(image_batch) File "/data/hliu/anaconda3/install/envs/segtran-master1/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/data/hliu/anaconda3/install/envs/segtran-master1/lib/python3.8/site-packages/nnunet/network_architecture/generic_UNet.py", line 400, in forward x = torch.cat((x, skips[-(u + 1)]), dim=1) RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 6 but got size 5 for tensor number 1 in the list.

Hi~please help me figure out some questions.

1. i found that " python3 train2d.py --task refuge --ds train,valid,test --split all --maxiter 10000 --net unet-scratch " should be " --task fundus", otherwise will report errors.
2. the data for polyp downloaded from https://github.com/DengPingFan/PraNet (search for "testing data") is not complete. some image is missing, i guess it should include in their training data. But for training data, they have several datasets mixed together into two folders(image and mask)..so should i manually select image to our folders? Could you have a look?

many thanks in advance

I got the error AttributeError: 'EasyDict' object has no attribute 'use_mince_transformer' as shown. I tried to fix it but it didn't work. Please tell me how to fix it. Thank you.

Thanks for your great project! I downloaded REFUGE dataset but the name and number of image files are different from names in this repo! For example, training part has 360 files with names 0001.jpg, ...

When I train polyp datasets, the error occured, logs as before:

10 epochs, 1002 itertations each.
  0%|                                          | 0/10 [00:00<?, ?it/s]

Image scales: 8x8. Voxels: [1, 1600, 1792]
outputs shape: torch.Size([1, 2, 320, 320])
mask_batch shape: torch.Size([1, 3, 320, 320])
  0%|                                          | 0/10 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "train2d.py", line 1100, in <module>
    mask_batch.permute([0, 2, 3, 1]))
  File "/root/anaconda3/envs/python377/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/anaconda3/envs/python377/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 617, in forward
    reduction=self.reduction)
  File "/root/anaconda3/envs/python377/lib/python3.7/site-packages/torch/nn/functional.py", line 2433, in binary_cross_entropy_with_logits
    raise ValueError("Target size ({}) must be the same as input size ({})".format(target.size(), input.size()))
ValueError: Target size (torch.Size([1, 320, 320, 3])) must be the same as input size (torch.Size([1, 320, 320, 2]))

Could you tell me how to solve this problem? thank u!