Hi,

I tried several runs of the S-BART w. Discourse&Action (last row of Table 2) on the SAMSum dataset, and the results are sometimes higher/lower than the BART baseline, which is probably caused by random seeds. Nevertheless, the performance gain on the ADSC dataset seems to be more substantial according to Table 3. I was wondering if it's possible to also release the code to test the model on that dataset?

Thanks.

Hello, Thanks for the nice paper and code! I notice that the decoder attends to every utterances after bart encoder. So I am wondering how to batch these, since the input to the model isList[torch.Tesnor] where len(list)=batch_size, torch.Tensor.shape=[num_utts_in_one_conversation, seq_len]?

In your code

loss_tensors = self._step(batch) # in SummarizationModule/train_step
outputs = self(src_ids, attention_mask=src_mask, decoder_input_ids=decoder_input_ids, use_cache=False, discourse_graph = self.hparams.discourse_graph, adj = adj, segmented_encoder = self.hparams.segmented_encoder, relation = self.hparams.relation, action_adj = action_adj, actions = actions, actions_mask = actions_mask,) # in SummarizationModule/_step

But it seems that BaseTransformer does not give forward ortrain_step or__call__ function? Thanks for your help

@jiaaoc Used versions: torch==1.3.0 pytorch-lightning==0.8.1 install rouge==1.0.0 transformers==3.2.0 --editable 'transformers/' Running bash script train_multi_graph.sh throws following error:

GPU available: True, used: True
INFO:lightning:GPU available: True, used: True
TPU available: False, using: 0 TPU cores
INFO:lightning:TPU available: False, using: 0 TPU cores
CUDA_VISIBLE_DEVICES: [0]
INFO:lightning:CUDA_VISIBLE_DEVICES: [0]
normal graph
dicsource_graph
Traceback (most recent call last):
  File "/Structure-Aware-BART-main/src/train.py", line 540, in <module>
    main(args)
  File "/Structure-Aware-BART-main/src/train.py", line 506, in main
    logger=logger,
  File "/Structure-Aware-BART-main/src/lightning_base.py", line 700, in generic_train
    trainer.fit(model)
  File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/trainer.py", line 918, in fit
    self.single_gpu_train(model)
  File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/distrib_parts.py", line 167, in single_gpu_train
    self.optimizers, self.lr_schedulers, self.optimizer_frequencies = self.init_optimizers(model)
  File "/usr/local/lib/python3.7/dist-packages/pytorch_lightning/trainer/optimizers.py", line 18, in init_optimizers
    optim_conf = model.configure_optimizers()
  File "/Structure-Aware-BART-main/src/lightning_base.py", line 181, in configure_optimizers
    new_params_id += list(map(id, model.model.discourse_encoder.parameters()))  +\
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 585, in __getattr__
    type(self).__name__, name))
AttributeError: 'BartModel' object has no attribute 'discourse_encoder'

Bash script train_base.sh throws the following error:

Global seed set to 42 Traceback (most recent call last): File "train.py", line 540, in <module> main(args) File "train.py", line 502, in main checkpoint_callback=get_checkpoint_callback( File "/export/home/dialogue-sum/sa-bart/src/callbacks.py", line 100, in get_checkpoint_callback checkpoint_callback = ModelCheckpoint( TypeError: __init__() got an unexpected keyword argument 'filepath'

Could be caused by an inconsistent pytorch_lightning version. The README does not specify which version the authors used.

when i run './train_base.sh'. I got the following error. Do you have any idea?

Traceback (most recent call last): File "train.py", line 543, in main(args) File "train.py", line 523, in main trainer.test() File "/home/wxu/anaconda3/envs/bart/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 712, in test results = self.__test_using_best_weights(ckpt_path, test_dataloaders) File "/home/wxu/anaconda3/envs/bart/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 724, in __test_using_best_weights 'ckpt_path is "best", but ModelCheckpoint is not configured to save the best model.' pytorch_lightning.utilities.exceptions.MisconfigurationException: ckpt_path is "best", but ModelCheckpoint is not configured to save the best model.

Hi,

I notice that in one earlier paper of yours, the decoder is BART-large, which leads to much better performance. Have you tried using BART-large in this paper? If not, why?

Thanks.

Script throws the following warning when running multi_graph training.

Some weights of BartForConditionalGeneration were not initialized from the model checkpoint at facebook/bart-base and are newly initialized: ['model.decoder.layers.0.resweight', 'model.decoder.layers.0.resweight_2', 'model.decoder.layers.0.discourse_attn.k_proj.weight', 'model.decoder.layers.0.discourse_attn.k_proj.bias', 'model.decoder.layers.0.discourse_attn.v_proj.weight', 'model.decoder.layers.0.discourse_attn.v_proj.bias', 'model.decoder.layers.0.discourse_attn.q_proj.weight', 'model.decoder.layers.0.discourse_attn.q_proj.bias', 'model.decoder.layers.0.discourse_attn.out_proj.weight', 'model.decoder.layers.0.discourse_attn.out_proj.bias', 'model.decoder.layers.0.discourse_attn_layer_norm.weight', 'model.decoder.layers.0.discourse_attn_layer_norm.bias', 'model.decoder.layers.0.action_attn.k_proj.weight', 'model.decoder.layers.0.action_attn.k_proj.bias', 'model.decoder.layers.0.action_attn.v_proj.weight', 'model.decoder.layers.0.action_attn.v_proj.bias', 'model.decoder.layers.0.action_attn.q_proj.weight', 'model.decoder.layers.0.action_attn.q_proj.bias', 'model.decoder.layers.0.action_attn.out_proj.weight', 'model.decoder.layers.0.action_attn.out_proj.bias', 'model.decoder.layers.0.action_attn_layer_norm.weight', 'model.decoder.layers.0.action_attn_layer_norm.bias', 'model.decoder.layers.0.composit_layer.weight', 'model.decoder.layers.0.composit_layer.bias', 'model.decoder.layers.0.composit_layer_norm.weight', 'model.decoder.layers.0.composit_layer_norm.bias', 'model.decoder.layers.1.resweight', 'model.decoder.layers.1.resweight_2', 'model.decoder.layers.1.discourse_attn.k_proj.weight', 'model.decoder.layers.1.discourse_attn.k_proj.bias', 'model.decoder.layers.1.discourse_attn.v_proj.weight', 'model.decoder.layers.1.discourse_attn.v_proj.bias', 'model.decoder.layers.1.discourse_attn.q_proj.weight', 'model.decoder.layers.1.discourse_attn.q_proj.bias', 'model.decoder.layers.1.discourse_attn.out_proj.weight', 'model.decoder.layers.1.discourse_attn.out_proj.bias', 'model.decoder.layers.1.discourse_attn_layer_norm.weight', 'model.decoder.layers.1.discourse_attn_layer_norm.bias', 'model.decoder.layers.1.action_attn.k_proj.weight', 'model.decoder.layers.1.action_attn.k_proj.bias', 'model.decoder.layers.1.action_attn.v_proj.weight', 'model.decoder.layers.1.action_attn.v_proj.bias', 'model.decoder.layers.1.action_attn.q_proj.weight', 'model.decoder.layers.1.action_attn.q_proj.bias', 'model.decoder.layers.1.action_attn.out_proj.weight', 'model.decoder.layers.1.action_attn.out_proj.bias', 'model.decoder.layers.1.action_attn_layer_norm.weight', 'model.decoder.layers.1.action_attn_layer_norm.bias', 'model.decoder.layers.1.composit_layer.weight', 'model.decoder.layers.1.composit_layer.bias', 'model.decoder.layers.1.composit_layer_norm.weight', 'model.decoder.layers.1.composit_layer_norm.bias', 'model.decoder.layers.2.resweight', 'model.decoder.layers.2.resweight_2', 'model.decoder.layers.2.discourse_attn.k_proj.weight', 'model.decoder.layers.2.discourse_attn.k_proj.bias', 'model.decoder.layers.2.discourse_attn.v_proj.weight', 'model.decoder.layers.2.discourse_attn.v_proj.bias', 'model.decoder.layers.2.discourse_attn.q_proj.weight', 'model.decoder.layers.2.discourse_attn.q_proj.bias', 'model.decoder.layers.2.discourse_attn.out_proj.weight', 'model.decoder.layers.2.discourse_attn.out_proj.bias', 'model.decoder.layers.2.discourse_attn_layer_norm.weight', 'model.decoder.layers.2.discourse_attn_layer_norm.bias', 'model.decoder.layers.2.action_attn.k_proj.weight', 'model.decoder.layers.2.action_attn.k_proj.bias', 'model.decoder.layers.2.action_attn.v_proj.weight', 'model.decoder.layers.2.action_attn.v_proj.bias', 'model.decoder.layers.2.action_attn.q_proj.weight', 'model.decoder.layers.2.action_attn.q_proj.bias', 'model.decoder.layers.2.action_attn.out_proj.weight', 'model.decoder.layers.2.action_attn.out_proj.bias', 'model.decoder.layers.2.action_attn_layer_norm.weight', 'model.decoder.layers.2.action_attn_layer_norm.bias', 'model.decoder.layers.2.composit_layer.weight', 'model.decoder.layers.2.composit_layer.bias', 'model.decoder.layers.2.composit_layer_norm.weight', 'model.decoder.layers.2.composit_layer_norm.bias', 'model.decoder.layers.3.resweight', 'model.decoder.layers.3.resweight_2', 'model.decoder.layers.3.discourse_attn.k_proj.weight', 'model.decoder.layers.3.discourse_attn.k_proj.bias', 'model.decoder.layers.3.discourse_attn.v_proj.weight', 'model.decoder.layers.3.discourse_attn.v_proj.bias', 'model.decoder.layers.3.discourse_attn.q_proj.weight', 'model.decoder.layers.3.discourse_attn.q_proj.bias', 'model.decoder.layers.3.discourse_attn.out_proj.weight', 'model.decoder.layers.3.discourse_attn.out_proj.bias', 'model.decoder.layers.3.discourse_attn_layer_norm.weight', 'model.decoder.layers.3.discourse_attn_layer_norm.bias', 'model.decoder.layers.3.action_attn.k_proj.weight', 'model.decoder.layers.3.action_attn.k_proj.bias', 'model.decoder.layers.3.action_attn.v_proj.weight', 'model.decoder.layers.3.action_attn.v_proj.bias', 'model.decoder.layers.3.action_attn.q_proj.weight', 'model.decoder.layers.3.action_attn.q_proj.bias', 'model.decoder.layers.3.action_attn.out_proj.weight', 'model.decoder.layers.3.action_attn.out_proj.bias', 'model.decoder.layers.3.action_attn_layer_norm.weight', 'model.decoder.layers.3.action_attn_layer_norm.bias', 'model.decoder.layers.3.composit_layer.weight', 'model.decoder.layers.3.composit_layer.bias', 'model.decoder.layers.3.composit_layer_norm.weight', 'model.decoder.layers.3.composit_layer_norm.bias', 'model.decoder.layers.4.resweight', 'model.decoder.layers.4.resweight_2', 'model.decoder.layers.4.discourse_attn.k_proj.weight', 'model.decoder.layers.4.discourse_attn.k_proj.bias', 'model.decoder.layers.4.discourse_attn.v_proj.weight', 'model.decoder.layers.4.discourse_attn.v_proj.bias', 'model.decoder.layers.4.discourse_attn.q_proj.weight', 'model.decoder.layers.4.discourse_attn.q_proj.bias', 'model.decoder.layers.4.discourse_attn.out_proj.weight', 'model.decoder.layers.4.discourse_attn.out_proj.bias', 'model.decoder.layers.4.discourse_attn_layer_norm.weight', 'model.decoder.layers.4.discourse_attn_layer_norm.bias', 'model.decoder.layers.4.action_attn.k_proj.weight', 'model.decoder.layers.4.action_attn.k_proj.bias', 'model.decoder.layers.4.action_attn.v_proj.weight', 'model.decoder.layers.4.action_attn.v_proj.bias', 'model.decoder.layers.4.action_attn.q_proj.weight', 'model.decoder.layers.4.action_attn.q_proj.bias', 'model.decoder.layers.4.action_attn.out_proj.weight', 'model.decoder.layers.4.action_attn.out_proj.bias', 'model.decoder.layers.4.action_attn_layer_norm.weight', 'model.decoder.layers.4.action_attn_layer_norm.bias', 'model.decoder.layers.4.composit_layer.weight', 'model.decoder.layers.4.composit_layer.bias', 'model.decoder.layers.4.composit_layer_norm.weight', 'model.decoder.layers.4.composit_layer_norm.bias', 'model.decoder.layers.5.resweight', 'model.decoder.layers.5.resweight_2', 'model.decoder.layers.5.discourse_attn.k_proj.weight', 'model.decoder.layers.5.discourse_attn.k_proj.bias', 'model.decoder.layers.5.discourse_attn.v_proj.weight', 'model.decoder.layers.5.discourse_attn.v_proj.bias', 'model.decoder.layers.5.discourse_attn.q_proj.weight', 'model.decoder.layers.5.discourse_attn.q_proj.bias', 'model.decoder.layers.5.discourse_attn.out_proj.weight', 'model.decoder.layers.5.discourse_attn.out_proj.bias', 'model.decoder.layers.5.discourse_attn_layer_norm.weight', 'model.decoder.layers.5.discourse_attn_layer_norm.bias', 'model.decoder.layers.5.action_attn.k_proj.weight', 'model.decoder.layers.5.action_attn.k_proj.bias', 'model.decoder.layers.5.action_attn.v_proj.weight', 'model.decoder.layers.5.action_attn.v_proj.bias', 'model.decoder.layers.5.action_attn.q_proj.weight', 'model.decoder.layers.5.action_attn.q_proj.bias', 'model.decoder.layers.5.action_attn.out_proj.weight', 'model.decoder.layers.5.action_attn.out_proj.bias', 'model.decoder.layers.5.action_attn_layer_norm.weight', 'model.decoder.layers.5.action_attn_layer_norm.bias', 'model.decoder.layers.5.composit_layer.weight', 'model.decoder.layers.5.composit_layer.bias', 'model.decoder.layers.5.composit_layer_norm.weight', 'model.decoder.layers.5.composit_layer_norm.bias', 'model.discourse_encoder.attention_0.W', 'model.discourse_encoder.attention_0.a', 'model.discourse_encoder.attention_0.one_hot_embedding.weight', 'model.discourse_encoder.attention_0.layer_norm.weight', 'model.discourse_encoder.attention_0.layer_norm.bias', 'model.discourse_encoder.attention_1.W', 'model.discourse_encoder.attention_1.a', 'model.discourse_encoder.attention_1.one_hot_embedding.weight', 'model.discourse_encoder.attention_1.layer_norm.weight', 'model.discourse_encoder.attention_1.layer_norm.bias', 'model.discourse_encoder.out_att.W', 'model.discourse_encoder.out_att.a', 'model.discourse_encoder.out_att.one_hot_embedding.weight', 'model.discourse_encoder.out_att.layer_norm.weight', 'model.discourse_encoder.out_att.layer_norm.bias', 'model.discourse_encoder.fc.weight', 'model.discourse_encoder.fc.bias', 'model.discourse_encoder.layer_norm.weight', 'model.discourse_encoder.layer_norm.bias', 'model.action_encoder.attention_0.W', 'model.action_encoder.attention_0.a', 'model.action_encoder.attention_0.layer_norm.weight', 'model.action_encoder.attention_0.layer_norm.bias', 'model.action_encoder.attention_1.W', 'model.action_encoder.attention_1.a', 'model.action_encoder.attention_1.layer_norm.weight', 'model.action_encoder.attention_1.layer_norm.bias', 'model.action_encoder.out_att.W', 'model.action_encoder.out_att.a', 'model.action_encoder.out_att.layer_norm.weight', 'model.action_encoder.out_att.layer_norm.bias', 'model.action_encoder.fc.weight', 'model.action_encoder.fc.bias', 'model.action_encoder.layer_norm.weight', 'model.action_encoder.layer_norm.bias'] You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.

Is this expected?