Hello, Thanks for sharing good research!

In the paper, figure 2, Binary Key-Patch Map is seem like used in BAG with input image features. I wonder how it worked in training. So, trying to find the part in the code, but I found this part in train.py.

https://github.com/jcwang123/BA-Transformer/blob/d1ccdff68beac82d18c2cd64bb800e51d7afc5e1/train.py#L189

It looks like point data(key-patch map or binary key-patch map) is used only for comparison with boundary prediction that came from the model. I think I missed the code that uses point data during the trained model. If it is, how the binary key-patch map is used in the training phase?

Thanks!

Hi, thanks for your work and your sharing~

For pre-processing the dataset, when I run python src/point_gen.py, I found there is no main function and no print information~ So, what can I do for the key-patch map generation?

thks~

Traceback (most recent call last): File "srcc/process_resize.py", line 87, in process_isic2018() File "srcc/process_resize.py", line 44, in process_isic2018 image_new = cv2.resize(image, dim, interpolation=cv2.INTER_CUBIC) TypeError: src data type = 17 is not supported

Hello,jcWnag, I follow your reademe to train but the following error occured. There is no validation data in ISIC2016. I just find the train and test dataset in the website. How can find the vaild dataset, or have other sulution? Thanks for your answer!

`class myDataset(data.Dataset): def init(self, split, aug=False): super(myDataset, self).init()

    self.image_paths = []
    self.label_paths = []
    self.point_paths = []
    self.dist_paths = []

    root_dir = './dataset/isic2016/'
    if split == 'train':
        self.image_paths = glob.glob(root_dir + '/Train/Image/*.npy')
        self.label_paths = glob.glob(root_dir + '/Train/Label/*.npy')
        self.point_paths = glob.glob(root_dir + '/Train/Point/*.npy')
    elif split == 'valid':
        self.image_paths = glob.glob(root_dir + '/Validation/Image/*.npy')
        self.label_paths = glob.glob(root_dir + '/Validation/Label/*.npy')
        self.point_paths = glob.glob(root_dir + '/Validation/Point/*.npy')
    elif split == 'test':
        self.image_paths = glob.glob(root_dir + '/Test/Image/*.npy')
        self.label_paths = glob.glob(root_dir + '/Test/Label/*.npy')
        self.point_paths = glob.glob(root_dir + '/Test/Point/*.npy')
    self.image_paths.sort()
    self.label_paths.sort()
    self.point_paths.sort()

    print('Loaded {} frames'.format(len(self.image_paths)))
    self.num_samples = len(self.image_paths)
    self.aug = aug

    self.transf = A.Compose([
        A.HorizontalFlip(p=0.5),
        A.VerticalFlip(p=0.5),
        A.RandomBrightnessContrast(p=0.2),
        A.Rotate()
    ])`

Hello, I recently read your paper, I think it is very good, but I found that formula (1) is missing the residual part of the MSA position, I don’t know if what I put forward is correct, and I hope you can take a look. thanks.

line 47: parser.add_argument('--ppl', type=int, default=6) What is the meaning of ppl parameter? I have checked that the BAT class and train.py in line 97: It should be determined whether used decoder, thought the type is int and not boolean.

Hello, author, I noticed that you mentioned in the article that all network encoders have been pre-trained on ImageNet, may I ask whether the encoders you mentioned refer to ResNet50 or ResNet50 + Transformer's encoder?

Also, as I understand it, in the BAT class there is a decoder section in Transformer regardless of which Transformer is used, so what is use_decoder for (although it seems you set it to False).

Thank you for the response

Thank you for your code, When I typed python train.py, I face the error as below following, how do we have the data_split.json? Thank you.

Thank you @jcwang123.

$ python train.py Namespace(arch='BAT', aug=1, bt_size=8, cross=0, dataset='isic2016', exp_name='', fold='0', gpu='0', lr_seg=0.0001, n_epochs=200, net_layer=50, patience=500, point_pred=1, ppl=6, pre=0, seg_loss=0, trans=1) Loaded 900 frames Loaded 379 frames pretrained resnet, 50 ERROR: Unexpected segmentation fault encountered in worker. Traceback (most recent call last): File "/home/mll/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 986, in _try_get_data