Hi @easezyc , thanks for your great implementations. When I'm trying RevGrad in pytorch1.0, I have some questions. Would you help me?

1. In the original paper, it said the optimizer was set as momentum=0.9. However, in line62, the optimizer will be created every iteration, which means momentum will be reset every time. https://github.com/easezyc/deep-transfer-learning/blob/cc97b7d248b7e7d9b187a3bae99eb560c458f89c/UDA/pytorch1.0/RevGrad/RevGrad.py#L62

2. The optimizer_critic seems do not optimizer_critic.step(). https://github.com/easezyc/deep-transfer-learning/blob/cc97b7d248b7e7d9b187a3bae99eb560c458f89c/UDA/pytorch1.0/RevGrad/RevGrad.py#L63

3. I tried to solve the questions, but I cannot reproduce the reported results. My modifications are below.

# defined optimizer_fea before the training loop
    optimizer_fea = torch.optim.SGD([
        {'params': model.sharedNet.parameters(), 'base_lr': lr/10},
        {'params': model.cls_fn.parameters(), 'base_lr': lr},
        {'params': model.domain_fn.parameters(), 'base_lr': lr}
        ], lr=lr, momentum=momentum, weight_decay=l2_decay)

. . .

for i in range(1, iteration+1):
         # update learning rate during training
        for param_group in optimizer_fea.param_groups:
            param_group['lr'] = param_group['base_lr'] / math.pow((1 + 10 * (i - 1) / iteration), 0.75)

Did I miss anything? Thanks for your help!

Hello, I was going through the paper DSAN and codebase. I was wondering could you make the TSNE visualization available as well. It will be helpful for me. I will cite your paper

Hi, thx for ur code.

There's an error when I run the DAN code.

The log can be seen here:

The only thing I edited in the code is changing the root_path from /data/zhuyc/OFFICE31/ to ./dataset/Original_images/(except for the print log with "tt" and "ttt"). I checked the size of source data and target data but nothing wrong, both are [32, 3, 224, 224]. Is there any mistake in the code or did I run the wrong way?

Thank you in advance and have a good day :>

您好前辈，非常感谢您的代码。

我有几个问题：

1. 在训练过程中，按理来说想使用全部数据的话一般是外面一层epoch循环，内里一层dataloader的循环。但是我看MFSAN里input只用了iterator的单次next()，所以每次epoch的源域和目标域的大小都只是一个batch_size吗？为什么不每次epoch将源域和目标域的所有图片都做迁移，是考虑到时间成本之类的问题吗？或者是采样的技巧吗？

2. 跟上个问题性质差不多，举个例子在MFSAN2中，按照您的代码的话，在第一个epoch中，是source1的第一个batch跟target的第一个batch去计算，然后source2的第一个batch是跟target的第二个batch去计算了。target不是同一个batch的话，会不会有些影响？

非常感谢您的回答，祝前辈中好多顶会 :>

if you are in a situation where soft-loss is equal to 0 and accuracy is 100%, you probably meet the same problem as me. You might need to check your dataset and ensure that sub-classes are under the category folder such as bike and back_pack instead of images. The correct data set layout is shown in the figure. I hope this works and can help you.

您好，非常棒的工作，谢谢您的开源分享！ 我在尝试复现您的工作DSAN，但是t-SNE出来的效果不太理想。 请问在A-->W这个任务上，您t-SNE可视化时选取的源域和目标域的样本分别是多少呢？ 我两个域都选取的是480个样本，但出来的图效果不好。 如果可以的话能否分享一下t-SNE的代码呢？ email: [email protected]

期待您的回复！谢谢！

Hi @easezyc ,

You provided both version of the implementation of RevGrad using Pytorch 0.3 and Pytorch 1.0.

In Pytorch 0.3 the code is like that `class RevGrad(nn.Module):

def __init__(self, num_classes=31):
    super(RevGrad, self).__init__()
    self.sharedNet = resnet50(False)
    self.cls_fc = nn.Linear(2048, num_classes)
    self.domain_fc = nn.Linear(2048, 2)

def forward(self, data):
    data = self.sharedNet(data)
    clabel_pred = self.cls_fc(data)
    dlabel_pred = self.domain_fc(data)

    return clabel_pred, dlabel_pred`

and in Pytorch 1.0 the code is like that:

class RevGrad(nn.Module):

    def __init__(self, num_classes=31):
        super(RevGrad, self).__init__()
        self.sharedNet = resnet50(True)
        self.cls_fn = nn.Linear(2048, num_classes)
        self.domain_fn = AdversarialNetwork(in_feature=2048)
             
    def forward(self, data):
        data = self.sharedNet(data)
        clabel_pred = self.cls_fn(data)
        dlabel_pred = self.domain_fn(AdversarialLayer(high_value=1.0)(data))
        #print(dlabel_pred)
        return clabel_pred, dlabel_pred



class AdversarialNetwork(nn.Module):
    def __init__(self, in_feature):
        super(AdversarialNetwork, self).__init__()
        self.ad_layer1 = nn.Linear(in_feature,1024)
        self.ad_layer2 = nn.Linear(1024,1024)
        self.ad_layer3 = nn.Linear(1024, 1)
        self.ad_layer1.weight.data.normal_(0, 0.01)
        self.ad_layer2.weight.data.normal_(0, 0.01)
        self.ad_layer3.weight.data.normal_(0, 0.3)
        self.ad_layer1.bias.data.fill_(0.0)
        self.ad_layer2.bias.data.fill_(0.0)
        self.ad_layer3.bias.data.fill_(0.0)
        self.relu1 = nn.ReLU()
        self.relu2 = nn.ReLU()
        self.dropout1 = nn.Dropout(0.5)
        self.dropout2 = nn.Dropout(0.5)
        self.sigmoid = nn.Sigmoid()

    def forward(self, x):
        x = self.ad_layer1(x)
        x = self.relu1(x)
        x = self.dropout1(x)
        x = self.ad_layer2(x)
        x = self.relu2(x)
        x = self.dropout2(x)
        x = self.ad_layer3(x)
        x = self.sigmoid(x)
        return x

    def output_num(self):
        return 1

class AdversarialLayer(torch.autograd.Function):
  def __init__(self, high_value=1.0):
    self.iter_num = 0
    self.alpha = 10
    self.low = 0.0
    self.high = high_value
    self.max_iter = 2000.0
    
  def forward(self, input):
    self.iter_num += 1
    output = input * 1.0
    return output

  def backward(self, gradOutput):
    self.coeff = np.float(2.0 * (self.high - self.low) / (1.0 + np.exp(-self.alpha*self.iter_num / self.max_iter)) - (self.high - self.low) + self.low)
    return -self.coeff * gradOutput

My question is which method is correct? If both methods are correct, can you please explain a bit. Thanks in advance.

You have done a great job, but I have a question about the MMD Loss.

You use the guassian kernel for computing, what are the kernel_mul and kernel_num mean in your code? Thank you very much.

In your paper, you have mentioned the discrepancy loss for aligning the classifiers. In your code I did not find the discrepancy loss for classifiers. Did I miss that? Can you please explain it?

Thank you in advanced.

Traceback (most recent call last): File "MRAN.py", line 131, in optimizer.param_group[0]['lr'] = args.lr[0] / math.pow((1 + 10 * (epoch - 1) / args.epochs), 0.75) AttributeError: 'SGD' object has no attribute 'param_group'

Hi,

First of all, thanks for the great work!

For DSAN, I am wondering if you can also release the source code of reproducing the T-SNE results. I tried the T-SNE pytorch version but the generated result is different from the one reported in the paper.

UPDATE: I saw there is a closed post that asking for t-SNE implementation and you want them to provide the email address. Would you please also send me a copy of code? My email address is: [email protected]

Thanks for your valuable time and I am looking forward to hearing from you soon!

Dear Authors

Thank you very much for your excellent work and for making the code publicly available. As you sent the code for the tSNE visualization to others, I am also requesting the code for MFSAN tSNE. I would really appreciate it. My email: is [email protected]

Thanks in advance