Hello, it appears that your DDI example (SciFive/finetune/re/ddi_1.ipynb) contain some errors. Could you please double-check that? For example, the authentication error in the line 'tensorflow gcs config.configure gcs from colab auth()' produces the following error:

Setting up GCS access...
Running on TPU: grpc://10.20.166.202:8470
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
[<ipython-input-3-8279b3908dbb>](https://localhost:8080/#) in <module>()
     16   auth.authenticate_user()
     17   tf.config.experimental_connect_to_host(TPU_ADDRESS)
---> 18   tensorflow_gcs_config.configure_gcs_from_colab_auth()
     19 
     20 tf.disable_v2_behavior()

[/usr/local/lib/python3.7/dist-packages/tensorflow_gcs_config/__init__.py](https://localhost:8080/#) in configure_gcs_from_colab_auth(device)
    130   adc_filename = os.environ.get(
    131       "GOOGLE_APPLICATION_CREDENTIALS", "/content/adc.json")
--> 132   with open(adc_filename) as f:
    133     data = json.load(f)
    134   return configure_gcs(credentials=data, device=device)

FileNotFoundError: [Errno 2] No such file or directory: '/content/adc.json'

Please let us know if you have any suggestion to fix the issue. Thank you in advance.

Hello, first of all, I want to say nice work!

When I want to reproduce your results on chemprot, I notice the following auth issue in the code

model.finetune(
    mixture_or_task_name="re_all",
    pretrained_model_dir=PRETRAINED_DIR,
    finetune_steps=FINETUNE_STEPS
)

2022-11-30 14:46:16.639835: W tensorflow/core/platform/cloud/google_auth_provider.cc:184] All attempts to get a Google authentication bearer token failed, returning an empty token. Retrieving token from files failed with "NOT_FOUND: Could not locate the credentials file.". Retrieving token from GCE failed with "FAILED_PRECONDITION: Error executing an HTTP request: libcurl code 6 meaning 'Couldn't resolve host name', error details: Could not resolve host: metadata".

Turns out that this is caused by not being able to find vocab which is in 'gs://t5-data/vocabs/cc_all.32000/sentencepiece.model'. But currently only gs://scifive is accessible.

Could you please release the vocab or share with us how exactly did you obtain the sentencepiece vocab so that we can reproduce the results? Thank you!

I am a PhD student trying to use your model for a research project.

Looking at the pre-training notebooks, it seems you do not use an init checkpoint to continue the training of the t5 model. Is this because you already have checkpoints in your model directory or because you train t5 from zero instead of using an already pre-trained T5 model?

How can I test the model in the QA task? Just input the text like " How many teeth do humans have?" or if i need to add the prefix,like "QA: How many teeth do humans have?"

Thanks for sharing the models. I found that the pubmed-pmc-large version (https://huggingface.co/razent/SciFive-large-Pubmed_PMC/tree/main) seems wrong as the fine-tuning results on MEDNLI are drastically worse than T5-large (acc 0.67+ vs 0.82+). However, the pubmed-large version gives good results.

Could you confirm this version?

Is it possible to share the fine-tuned MedNLI classifier with a simple code snippet for how to perform inference given a text and premise pair? I saw the notebook for fine-tuning but was wondering if the output could be open sourced. many thanks

Hi there,

we want to convert your tensorflow model to pytorch model. We need the config file to do this, however we did not find the config file from this repo, https://console.cloud.google.com/storage/browser/scifive/models?pageState=(%22StorageObjectListTable%22:(%22f%22:%22%255B%255D%22))&prefix=&forceOnObjectsSortingFiltering=false

could you please share the config file also?

I am trying out the code in scifive_pretrain_base.ipynb.

I got OSError: Unable to open file for all the gs: files in the code 'gs://t5_training/t5-data/config/pretrained_models_google_base_operative_config.gin' gs://mindxhack/bio_sentence_piece_small.txt

I tried look them up using the google cloud storage browser and don't see these files.

The browser does find the model files like gs://scifive/models/pubmed_pmc/base

So the question is whether this is working code as is. Do these dependent files still exist on the cloud?

Hi, Thanks for your great contribution in biomedical domain. I tried all the models in the hugging face format and I couldn't replicate any of the results or even get a reasonable output. Is there something wrong with the code, model, or anything is missing?

I run the following code:

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("razent/SciFive-base-PMC")  
model = AutoModelForSeq2SeqLM.from_pretrained("razent/SciFive-base-PMC")
model.to(device)
sentence = "Identification of APC2 , a homologue of the adenomatous polyposis coli tumour suppressor ."
text =  "ncbi_ner: " + sentence + " </s>"

encoding = tokenizer.encode_plus(text, pad_to_max_length=True, return_tensors="pt")
input_ids, attention_masks = encoding["input_ids"].to(device), encoding["attention_mask"].to(device)

outputs = model.generate(
    input_ids=input_ids, attention_mask=attention_masks,
    max_length=256,
    early_stopping=True
)

for output in outputs:
    line = tokenizer.decode(output, skip_special_tokens=True, clean_up_tokenization_spaces=True)
    print(line)

And this is the output:

ncbi_ner: ncbi_ner: ncbi_ner:

The expected output (based on the paper) should be as follow:

Identification of APC2 , a homologue of the entity* adenomatous polyposis coli tumour *entity suppressor .

I replaced the model with all other available large, base, pubmed, pmc, pubmed+pmc models (basically all 6 hugging face variations) but I didn't get any reasonable outputs.

Could you give me a solution?