Hi, It's a very interesting and solid study. And I'm trying to reproduce it. But when I run "bash src/scripts/preprocess_data.sh" in the "cc/main" folder, I get stuck. The process hangs here for two days:

I think line 112 (in function separate_dps) of cc/main/src/utils/utils.py should be

        aug_asts.append([ast[i : i + max_len], i + half_len])

instead of

        aug_asts.append([ast[i : i + max_len], half_len])

Hello, I have not found there exists a mask matrix in encoder for code completion, so every node of ast will participate in self attention computation?

Thanks. Another question is the default setting of eval_part for run_all.py is 'test', so does it means that you use the test set to evaluate the checkpoint during model training? It seems a lot wired. best wishes

Greetings! Congrats for the great work! Is the anonymize operation only for user-defined variable names? Will the API be anonymized? If so, will the semantics be changed?

Hello, your work is very good. I'd like to ask where the code of this article "a simple approach for handling out of volatile identifiers in deep learning for source code" is. Thank you!

We also suggest adding flag --anonymize order to the generated command for the VM task, in order to use our anonymization of the out-of-vocabulary identifiers. This simple technique will increase the test quality by several percent.

but i do not find --anonymize in run_all.py

From my understanding, mrr function uses the variable where to filter out tokens in memory (e.g., the first half tokens for intermediate blocks as explained in #4 ) , unknown tokens, and padding tokens. Then _mrr function sorts the top-10 predicted tokens for each position and uses the indexes of correctly predicted tokens of each position to calculate MRR.

My question is are all positions in this code block predicted at the same time or they are all predicted when calculating MRR (i.e., in _step() function of LightningModel class)? Let's say this code block is [range(250, 750), 250], then the range(250, 500) is the memory, and range(500, 750) is to be predicted. So when we calculate MRR of this block, is it for all the tokens in the range(500, 750)?

How does it complete the code block? Does it predict the 501th token based on range(250, 500) and then predict 502nd token based on range(250, 501)? Could you explain more and point out the related code.

Besides, how can I use a trained model for inference and get the vocab from their idx? Here is the script I wrote. I'm not sure about it. Please let me know your idea. Thanks for your help!

...
args.load = 'path to the trained model.ckpt'

model = LightningModel(args)
model.eval()

# use eval_dataset for example
eval_dataset = model.eval_dataset
eval_dataloader = DataLoader(eval_dataset, 
    num_workers=args.num_workers, collate_fn=model.collate_fn, drop_last=False, pin_memory=True, shuffle=False)

# get the vocab of tensor idx
def idx2vocab_func(batch, idx2vocab): # assuming batch = 1
    vocab_len = len(idx2vocab)
    batch = batch.view(-1).tolist()
    input_v = list()
    for i in batch:
        if i >= vocab_len:
            input_v.append('UNK')
        else:
            input_v.append(idx2vocab[i])
    return input_v

def get_pred(batch):
    y = batch['input_seq']['values']
    y_pred_types, y_pred_values = model(batch['input_seq'], rel=batch['rel_mask'], positions=batch['positions'])
    
    ext = batch['extended'].unsqueeze(-1).repeat(1, y.size(-1))
    ext_ids = torch.arange(y.size(-1), device=ext.device).view(1, -1).repeat(*(y.size()[:-1]+(1,)))
    where = ext_ids >= ext
    where = where.view(-1)

    y_pred_values = y_pred_values.view(-1, y_pred_values.size(-1))[where]

    _, y_pred_values = torch.topk(y_pred_values, k=1, dim=-1) # choose the top1 predicted token
    return y_pred_values.view(-1)

with torch.no_grad():
    for i, sample in enumerate(eval_dataloader):
        print('---------------Input----------------')
        input_v = sample['input_seq']['values']
        print(input_v.shape)
        print('Vocab:', ' '.join(idx2vocab_func(input_v, idx2vocab_value)))

        print('---------------Target----------------')
        target_v = sample['target_seq']['values']
        print(target_v.shape)
        print('Vocab:', ' '.join(idx2vocab_func(target_v, idx2vocab_value)))

        print('---------------Predicted----------------')
        pred_v = get_pred(sample)
        print(pred_v.shape)
        print('Vocab:', ' '.join(idx2vocab_func(pred_v, idx2vocab_value)))

Greetings! Congrats for the great work! My question is when you will post the instructions for reproducing the experients from the OOV anonymization paper. I believe that will be of great help to those who follow this work. It is a very interesting work and I think that it disearves more attention. Thanks in advance!