Hi, amazing work! and really appreciate you sharing the code.

I was checking it and there is a typo in norm-analysis-of-transformer/exp2_nmt/train.sh, in --trainpref $DATS_DIR/train.bpe it should be $DATA_DIR

#!/bin/sh

export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8

DATA_DIR=./work/processed_data
BPE_DIR=./work/bpe_model_and_vocab

# convert BPE vocab to use for fairseq
cut -f1 $BPE_DIR/de.vocab | tail -n +4 | sed "s/$/ 100/g" > $DATA_DIR/de.vocab
cut -f1 $BPE_DIR/en.vocab | tail -n +4 | sed "s/$/ 100/g" > $DATA_DIR/en.vocab

# fairseq preprocess
fairseq-preprocess --source-lang de --target-lang en \
    --trainpref $DATS_DIR/train.bpe \
    --validpref $DATA_DIR/valid.bpe \
    --testpref $DATA_DIR/valid.bpe \
    --srcdict $DATA_DIR/de.vocab \
    --tgtdict $DATA_DIR/en.vocab \
    --destdir $DATA_DIR/fairseq_preprocessed_data \
    --workers 16 # adjust here to suit your environment

Hi,

I'm trying to reproduce the results in the demo notebook emnlp2020/changed_fairseq_usage.ipynb with the hope of extending them to analyse a BART model. However, I'm currently running into issues when trying to load the model.

Here's the traceback:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-0ba6c5e0035c> in <module>
      6     model_name_or_path=path,
      7     tokenizer='moses',
----> 8     bpe='fastbpe',
      9 ).to(device)
     10 

~/INSTALLS/norm-analysis-of-transformer/emnlp2020/fairseq/fairseq/models/fairseq_model.py in from_pretrained(cls, model_name_or_path, checkpoint_file, data_name_or_path, **kwargs)
    275             data_name_or_path,
    276             archive_map=cls.hub_models(),
--> 277             **kwargs,
    278         )
    279         logger.info(x["args"])

~/INSTALLS/norm-analysis-of-transformer/emnlp2020/fairseq/fairseq/hub_utils.py in from_pretrained(model_name_or_path, checkpoint_file, data_name_or_path, archive_map, **kwargs)
     71     models, args, task = checkpoint_utils.load_model_ensemble_and_task(
     72         [os.path.join(model_path, cpt) for cpt in checkpoint_file.split(os.pathsep)],
---> 73         arg_overrides=kwargs,
     74     )
     75 

~/INSTALLS/norm-analysis-of-transformer/emnlp2020/fairseq/fairseq/checkpoint_utils.py in load_model_ensemble_and_task(filenames, arg_overrides, task, strict, suffix, num_shards)
    285             if not PathManager.exists(filename):
    286                 raise IOError("Model file not found: {}".format(filename))
--> 287             state = load_checkpoint_to_cpu(filename, arg_overrides)
    288             if "args" in state and state["args"] is not None:
    289                 cfg = convert_namespace_to_omegaconf(state["args"])

~/INSTALLS/norm-analysis-of-transformer/emnlp2020/fairseq/fairseq/checkpoint_utils.py in load_checkpoint_to_cpu(path, arg_overrides)
    237         overwrite_args_by_name(state["cfg"], arg_overrides)
    238 
--> 239     state = _upgrade_state_dict(state)
    240     return state
    241 

~/INSTALLS/norm-analysis-of-transformer/emnlp2020/fairseq/fairseq/checkpoint_utils.py in _upgrade_state_dict(state)
    458             state["args"].post_process = state["args"].remove_bpe
    459 
--> 460         state["cfg"] = convert_namespace_to_omegaconf(state["args"])
    461 
    462     if "cfg" in state and state["cfg"] is not None:

~/INSTALLS/norm-analysis-of-transformer/emnlp2020/fairseq/fairseq/dataclass/utils.py in convert_namespace_to_omegaconf(args)
    296 
    297 
--> 298     with initialize(config_path=config_path, strict=True): # BUG: TypeError: __init__() got an unexpected keyword argument 'strict'
    299     # with initialize(config_path=config_path):
    300         # import pdb;pdb.set_trace()

TypeError: __init__() got an unexpected keyword argument 'strict'

This seems to me to be an issue with the dependencies, so I've tried a few different versions of Hydra (e.g. 1.0.0, 1.1.0, 1.2.0) and dropping back to pytorch=1.7.1, but still no luck.

I've attached the list of installed dependencies after following the setup instructions in the README. Can you provide some details on the versions used to run this notebook?

Thanks in advance! Tannon

default_versions.txt