when i run "python run_expid.py --config /..../ --expid 191 --gpu 0" in my code,i get 2 results log .they are ..._tune_191_ahdfy68.log and ..._tune_051_os191usd.log.In second log file name,it also has the string of '191'.

I am not sure whether the implementation of DIN here is consistent with the original official released code DIN or DIEN. Given query_field (e.g., query Goods ID, query Cate ID) and history_field (e.g., Goods ID, Cate ID), in the original implementation, they first concatenate the embeddings of history_field (i.e., [Goods ID embedding, Cate ID embedding]) and then perform so-called "local activation unit (LAU)" with the concatenated query_field (i.e., [query Goods ID embedding, query Cate ID embedding]). In contrast, the implementation here seems to first perform the LAU then do the concatenation. Specifically, the code in DIN.py:

# 1. perform the LAU for each history_field
for idx, (din_query_field, din_history_field) \
            in enumerate(zip(self.din_query_field, self.din_history_field)):
            item_emb = feature_emb_dict[din_query_field]
            history_sequence_emb = feature_emb_dict[din_history_field]
            pooled_history_emb = self.attention_layers[idx](item_emb, history_sequence_emb)
            feature_emb_dict[din_history_field] = pooled_history_emb
# 2. do the concatenation here
feature_emb = self.embedding_layer.dict2tensor(feature_emb_dict)

If my understanding is correct, I wonder whether this will affect the final performance compared with the original implementation?

HFM adopts HolographicInteractionLayer in fuxictr/pytorch/layers/interaction.py.

However, HolographicInteractionLayer may be not available in Pytorch 1.10 because torch.rfft/torch.irfft has changed.\

Here is a solution for reference:

try:
    from torch import irfft
    from torch import rfft
except ImportError:
    from torch.fft import irfft2
    from torch.fft import rfft2
    def rfft(x, d):
        t = rfft2(x, dim = (-d))
        return torch.stack((t.real, t.imag), -1)
    def irfft(x, d, signal_sizes):
        return irfft2(torch.complex(x[:,:,0], x[:,:,1]), s = signal_sizes, dim = (-d))

Hi. Thank you for the nice RecSys code.

I am sending you PR because there was a mistake in the InterHAT code. I think it should be modified in MultiHeadSelfAtt code :

• before : value = value(batch_size, -1, self.num_heads, self.head_dim).transpose(1, 2)
• after : value = value.view(batch_size, -1, self.num_heads, self.head_dim).transpose(1, 2)

Please checking this PR!

Thanks.

For StandardScaler, looks like it supports NaN values, see class Normalizer:

null_index = np.isnan(X)

However, during preprocess, _fill_na() will fill na_value for non-string. So

• for dtype=str, the X values will be string
• for dtype=float/int, the X values will be na_value

In the first case, np.isnan will throw an error because X elements are of string type. In the second case, there is no point to normalize numbers if we have a na_value there.

Is this behavior expected or not?

Should field "sequence" share embeddings with the field "adgroup_id"? I found that the method "encoder.fit()" assigns the encoder such as tokenizer for each field. Since the given tiny datasets record the user historical behavior (ad sequence), then in my understanding that the id that appeared in the field "sequence" may also appear in the field "adgroup_id". As a result, it seems that the field "sequence" should share the same encoder (i.e., tokenizer) with the field "adgroup_id", but the demo "DeepFM_with_sequence_feature.py" gives separate encoders for these two fields.

41/5000 When I try to use sequence characteristics, the program has an array out of bounds error. Could you please give an example of sequence features.

The code of splitting heads is buggy, without transpose the shape. The buggy code impacts AutoInt, DESTINE, InterHAt

query = query.view(batch_size * self.num_heads, -1, self.attention_dim)
key = key.view(batch_size * self.num_heads, -1, self.attention_dim)
value = value.view(batch_size * self.num_heads, -1, self.attention_dim)

The buggy code is from the reference code https://zhuanlan.zhihu.com/p/47812375

The correct code should be:

query = query.view(batch_size, seq_len, self.num_heads, self.attention_dim).transpose(1, 2)
key = key.view(batch_size, seq_len, self.num_heads, self.attention_dim).transpose(1, 2)
value = value.view(batch_size, seq_len, self.num_heads, self.attention_dim).transpose(1, 2)

The paper uses concat(LN(e1), LN(e2),..., LN(ef))， but the code use nn.LayerNorm([feature_map.num_fields, embedding_dim]). This makes normalization happen in the last two dimensions.

This PR adds the option to add sample weights to model instances, by specifying weight_col to the dataset config. This creates a sample_weight: 1 in the feature map json. When this is absent, from now on all instances receive a weight 1 by default, since pytorch enforces DataLoader to output tensor so we cannot set the weights to None. We also set the package dependency to torch=1.7 the last version that supports fft as used now by the HolographicInteractionLayer.