Hi, When I run the test.py , I get the following error:

` File "/home/wf/anaconda3/envs/trunet/lib/python3.7/site-packages/ml_collections/config_dict/config_dict.py", line 883, in getitem field = self._fields[key] KeyError: 'skip_channels'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/wf/anaconda3/envs/trunet/lib/python3.7/site-packages/ml_collections/config_dict/config_dict.py", line 807, in getattr return self[attribute] File "/home/wf/anaconda3/envs/trunet/lib/python3.7/site-packages/ml_collections/config_dict/config_dict.py", line 889, in getitem raise KeyError(self._generate_did_you_mean_message(key, str(e))) KeyError: "'skip_channels'"

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/wf/anaconda3/envs/project_TransUNet_2/project_TransUNet/TransUNet/test.py", line 118, in net = ViT_seg(config_vit, img_size=args.img_size, num_classes=config_vit.n_classes).cuda() File "/home/wf/anaconda3/envs/project_TransUNet_2/project_TransUNet/TransUNet/networks/vit_seg_modeling.py", line 378, in init self.decoder = DecoderCup(config) File "/home/wf/anaconda3/envs/project_TransUNet_2/project_TransUNet/TransUNet/networks/vit_seg_modeling.py", line 343, in init skip_channels = self.config.skip_channels File "/home/wf/anaconda3/envs/trunet/lib/python3.7/site-packages/ml_collections/config_dict/config_dict.py", line 809, in getattr raise AttributeError(e) AttributeError: "'skip_channels'"`

Then I change skip_channels = self.config.skip_channels (in vit_seg_modeling.py 343 line) to skip_channels = [512,256,64,16] , I get the following new error:

RuntimeError: Error(s) in loading state_dict for VisionTransformer: Unexpected key(s) in state_dict: "transformer.embeddings.hybrid_model.root.conv.weight", size mismatch for transformer.embeddings.patch_embeddings.weight: copying a param with shape torch.Size([768, 1024, 1, 1]) from checkpoint, the shape in current model is torch.Size([768, 3, 16, 16])

For me, the cause of this is unclear. Does anyone have an idea?

First, most of your code is taken from(https://github.com/jeonsworld/ViT-pytorch) which is owned by @jeonsworld. Second, the entire idea of your paper is taken from (https://arxiv.org/abs/2012.15840). In your paper, you have not mentioned that they are the first to propose this architecture and your work is derived from them.

This is very unprofessional. I hope the famous people in your paper already knows about your conduct.

Thanks for your great contribution！ Would u like to share your image preprocess code, that is get 3D image normalize and ertract 2D from 3D and then save to .npz file Thanks !

@Beckschen , Hello, I am trying to extend an existing continual semantic segmentation method to be suitable for 3D medical datasets. The Synapse dataset (Medical Image Segmentation on Synapse multi-organ CT) seems to be a good choice for us to construct further experiments. However, we can not achieve satisfactory baseline performance due to the lack of 3D data preprocessing. It will be very nice if you would like to share the preprocessed data. My email: [email protected]

Thanks for your work. I have some problems about using it for another task. I modified the dataloader, my input are {img (1,3,512,512), label(1,1,512,512) }, but on training phase, I got the output=model(image_patch) (1,2,512,512), when calc loss, caused error RuntimeError: 1only batches of spatial targets supported (3D tensors) but got targets of size: : [1, 1, 512, 512].

args: vit_name = R50-ViT-B_16, n_skip=3, vit_patches_size=16

can you give me some advice? thank you!

Hello, I use the preprocessed data (requested as mentioned in the Readme) and get the following error:

TransUNet$ python test.py --dataset Synapse --vit_name R50-ViT-B_16 Traceback (most recent call last): File "test.py", line 12, in from datasets.dataset_synapse import Synapse_dataset ModuleNotFoundError: No module named 'datasets.dataset_synapse'

For me, the cause of this is unclear. Does anyone have an idea?

In the original implementation, the position embedding has a dimension of [ n_patches+1, hidden_size] to accommodate for additional class token:

https://github.com/jeonsworld/ViT-pytorch/blob/a786151f6ceed00e97ab526772916faec5efb8ed/models/modeling.py#L150

In your implementation, you removed the class token and your position embedding has a dimention [ n_patches, hidden_size]:

https://github.com/Beckschen/TransUNet/blob/main/networks/vit_seg_modeling.py#L149

When I tried to load the pre-trained model based on your changes (the new size) , I get a mismatch error:

model.load_from(np.load(args.pretrained_dir)), in load_from self.transformer.embeddings.position_embeddings.copy_(np2th(posemb)) RuntimeError: The size of tensor a (196) must match the size of tensor b (170) at non-singleton dimension 1

Would you please explain how the pretrained checkpoint can be loaded based on these changes ?

Hi! I am very interested in your paper and solution for the medical image segmentation.

Regarding my first question, I ran the test.py and it outputs that there is a problem in line 122 in the code and that there is no such file or directory '../model/TU_Synapse224/TU_pretrain_R50-ViT-B_16_skip3_bs24_224/epoch_29.pth' I hope that you can help me with this problem. I did run it on single image though, if that can be some sort of an issue.

Second question is, why is there a need to clip an image between [-125, 275] ? Image that I downloaded is grayscale of the uint8 data type.

TransUNet/networks/vit_seg_modeling_resnet_skip.py)

When I tried to add the 4th channel but it throws me this error:

RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[2, 4, 256, 256] to have 3 channels, but got 4 channels instead

Hello, thanks a lot for your work. I have a question about the testing result on Synapse dataset. I have trained the network with R50-ViT-B_16, and base_lr=0.01, batch_size=24,n_skip=3, img_size=224, max_epochs=300, vit_patches_size=16. The difference with method in paper is: num_classes=14, because there 13 classes and background in Synapse dataset. After 300 epochs, I use <python test.py --dataset Synapse --vit_name R50-ViT-B_16 --max_epochs 300> to test. However, I got <mean_dice : 0.131854 mean_hd95 : 83.012894>, this is far from the result in paper. I don't know the reason. Can somebody help me?

I got below error. Any idea please.

<main.Args object at 0x000002995A0011C0> <main.Args object at 0x000002995A0011C0> The length of train set is: 2211 93 iterations per epoch. 13950 max iterations 93 iterations per epoch. 13950 max iterations 0%| | 0/150 [00:00<?, ?it/s]

AttributeError Traceback (most recent call last) C:\Users\60962~1.LPI\AppData\Local\Temp/ipykernel_19332/2385906778.py in 90 91 trainer = {'Synapse': trainer_synapse,} ---> 92 trainer[dataset_name](args, net, snapshot_path)

C:\Users\60962~1.LPI\AppData\Local\Temp/ipykernel_19332/1972247546.py in trainer_synapse(args, model, snapshot_path) 50 iterator = tqdm(range(max_epoch), ncols=70) 51 for epoch_num in iterator: ---> 52 for i_batch, sampled_batch in enumerate(trainloader): 53 image_batch, label_batch = sampled_batch['image'], sampled_batch['label'] 54 image_batch, label_batch = image_batch.cuda(), label_batch.cuda()

D:\anaconda3\lib\site-packages\torch\utils\data\dataloader.py in iter(self) 366 return self._iterator 367 else: --> 368 return self._get_iterator() 369 370 @property

D:\anaconda3\lib\site-packages\torch\utils\data\dataloader.py in _get_iterator(self) 312 else: 313 self.check_worker_number_rationality() --> 314 return _MultiProcessingDataLoaderIter(self) 315 316 @property

D:\anaconda3\lib\site-packages\torch\utils\data\dataloader.py in init(self, loader) 925 # before it starts, and del tries to join but will get: 926 # AssertionError: can only join a started process. --> 927 w.start() 928 self._index_queues.append(index_queue) 929 self._workers.append(w)

D:\anaconda3\lib\multiprocessing\process.py in start(self) 119 'daemonic processes are not allowed to have children' 120 _cleanup() --> 121 self._popen = self._Popen(self) 122 self._sentinel = self._popen.sentinel 123 # Avoid a refcycle if the target function holds an indirect

D:\anaconda3\lib\multiprocessing\context.py in _Popen(process_obj) 222 @staticmethod 223 def _Popen(process_obj): --> 224 return _default_context.get_context().Process._Popen(process_obj) 225 226 class DefaultContext(BaseContext):

D:\anaconda3\lib\multiprocessing\context.py in _Popen(process_obj) 325 def _Popen(process_obj): 326 from .popen_spawn_win32 import Popen --> 327 return Popen(process_obj) 328 329 class SpawnContext(BaseContext):

D:\anaconda3\lib\multiprocessing\popen_spawn_win32.py in init(self, process_obj) 91 try: 92 reduction.dump(prep_data, to_child) ---> 93 reduction.dump(process_obj, to_child) 94 finally: 95 set_spawning_popen(None)

D:\anaconda3\lib\multiprocessing\reduction.py in dump(obj, file, protocol) 58 def dump(obj, file, protocol=None): 59 '''Replacement for pickle.dump() using ForkingPickler.''' ---> 60 ForkingPickler(file, protocol).dump(obj) 61 62 #

AttributeError: Can't pickle local object 'trainer_synapse..worker_init_fn'

Dear researchers, please also consider checking our unet implementation of the proposed focal modulation block for small datasets problems, specially in medical domains. Our model shows the state of the art result camparing to base swin-unet and transunet models in terms of dice score on synapse dataset. https://github.com/givkashi/Focal-Unet Thanks.