Hi, sorry to bother you. I'm having trouble saving the training model file and loading the model. I can't load the files I trained and saved in the output folder, _checkpoint.bin and _checkpoint_adv.bin. I want to load the trained model to do some visualization experiments, but I am confused about the .bin file. I can load ViT-B_16.npz normally for Attention map visualization. Can you share how you loaded the .bin file for Attention map visualization?

here is the code to save model in your train.py.

`def save_model(args, model, is_adv=False):

model_to_save = model.module if hasattr(model, 'module') else model if not is_adv: model_checkpoint = os.path.join(args.output_dir, args.dataset, "%s_checkpoint.bin" % args.name) else: model_checkpoint = os.path.join(args.output_dir, args.dataset, "%s_checkpoint_adv.bin" % args.name) torch.save(model_to_save.state_dict(), model_checkpoint) logger.info("Saved model checkpoint to [DIR: %s]", os.path.join(args.output_dir, args.dataset))`

I'm a newbie, so I'm very sorry to bother you with this question.

Hi, thank you very much for publishing the code I've found your paper really interesting and was trying to figure if this can be adapted to my own model (I'm using basically the same configuration as yours). The paper highlights two main components DCM and TAMand i was wondering if this two modules can be extracted from the code.

1. Looking at the code I get that DCM is just a classification head which is performing a classification task on the domain, as DANN (I also see GRL). Am I correct?

2. I am having a harder life finding what is called TAM. Is it something that is built inside the blocks? Because I see that the main change is that the adversarial net is used inside each block to generate a loss_ad . Is this what you call patch-level discriminator inside the paper? Is there need to add something else to create the so called TAM

Thank you very much again for sharing the code

I try to run your code, but get the error as title, and I found there is no file named 'modeling_resnet'(in the code 'import .modeling_resnet import ResNetV2'), are there any file lost? Thanks.

Hello, I'm very happy to read your article. I have some puzzles. If you have time, can you help me to solve them?

How to realize the feature visualization in your article? What format is the data?

Hi！

Recently, when I was running your method, I encountered a problem. When I used baseline, the performance of all tasks whose target domain is Clipart decreased greatly. Can you help me solve this problem? It's more important to me.

Hello, I encountered something I didn't understand in the process of reading the code. If you have time, can you help me point it out? At line 99 of the code/modeling.py, this is： if posi_emb is not None: eps=1e-10 batch_size = key_layer.size(0) patch = key_layer ad_out, loss_ad = lossZoo.adv_local(patch[:,:,1:], ad_net, is_source) entropy = - ad_out * torch.log2(ad_out + eps) - (1.0 - ad_out) * torch.log2(1.0 - ad_out + eps) entropy = torch.cat((torch.ones(batch_size, self.num_attention_heads, 1).to(hidden_states.device).float(), entropy), 2) trans_ability = entropy if self.vis else None # [B12197] entropy = entropy.view(batch_size, self.num_attention_heads, 1, -1) attention_probs = torch.cat((attention_probs[:,:,0,:].unsqueeze(2) * entropy, attention_probs[:,:,1:,:]), 2) What I don't understand is, is this place a place to fight against discrimination losses? Which part of Vit did you improve?

Hi, thanks for releasing the open-source project of transformer-based UDA! May I ask you some questions?

I tried to reproduce the Baseline result, which denotes vanilla ViT with adversarial adaptation. Differently, I fixed the bs as 32, the input size as 224 * 224. Accordingly, I got the results on the setting of office31, W@A， D@A. [1] Safe Self-Refinement for transformer-based Domain Adaptation (https://arxiv.org/pdf/2204.07683.pdf) Obviously, [1] uses the Timm library and gets a very strong baseline. Have you thought about switching to Timm to improve performance? It's hard to compare because everyone implements baseline in different ways and the results vary greatly.:sweat_smile:

Additionally, I had a problem with the training baseline mentioned before. Model performance rises and then deteriorates rapidly. The training parameters are as follows: --beta 0.1 --gamma 0. --theta 0. Have you ever encountered such a problem? Can you give me some advice?

I run some script in ‘script.txt’, but get result far away from result reported in paper. Here are some result: Office-home Pr->Cl:52.37 Cl->Pr:78.64 Cl->Ar:72.39 Office-31 AD:94.98 AW:94.21 DW:76.78 Visda17 Train->Val:80.78

ps: apex didn't work well for some installation error, but i think it have no effect on result.(have effect on GPU memory and accumulation.) （【fused_weight_gradient_mlp_cuda module not found. gradient accumulation fusion with weight gradient computation disabled.】 I didn't find a resolution for this error）