Inside the 'model_train_upload_workflow.ipynb' file

We have the ability to upload to huggingface. However I do not have the file 'HuggingFace-API-key.txt'. Do I make the file myself? In which folder? I followed along the YT tutorial and she explains it differently than the code written. Can someone explain how they pushed it to huggingface plz.

When running the main function step I am getting the above error in the title. Stack trace is below. I am attempting to run a medium model.

Any help is appreciated!

08/31/2021 02:34:47 - WARNING - __main__ -   Process rank: -1, device: cuda, n_gpu: 1, distributed training: False, 16-bits training: False
/usr/local/lib/python3.7/dist-packages/transformers/models/auto/modeling_auto.py:902: FutureWarning: The class `AutoModelWithLMHead` is deprecated and will be removed in a future version. Please use `AutoModelForCausalLM` for causal language models, `AutoModelForMaskedLM` for masked language models and `AutoModelForSeq2SeqLM` for encoder-decoder models.
  FutureWarning,
08/31/2021 02:35:03 - INFO - __main__ -   Training/evaluation parameters <__main__.Args object at 0x7fd830413090>
08/31/2021 02:35:03 - INFO - __main__ -   Creating features from dataset file at cached
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-26-523c0d2a27d3> in <module>()
----> 1 main(trn_df, val_df)

10 frames
<ipython-input-25-aa20b6fc78bc> in main(df_trn, df_val)
     61     # Training
     62     if args.do_train:
---> 63         train_dataset = load_and_cache_examples(args, tokenizer, df_trn, df_val, evaluate=False)
     64 
     65         global_step, tr_loss = train(args, train_dataset, model, tokenizer)

<ipython-input-15-67f62bb60333> in load_and_cache_examples(args, tokenizer, df_trn, df_val, evaluate)
      2 
      3 def load_and_cache_examples(args, tokenizer, df_trn, df_val, evaluate=False):
----> 4     return ConversationDataset(tokenizer, args, df_val if evaluate else df_trn)
      5 
      6 

<ipython-input-14-a654172287f5> in __init__(self, tokenizer, args, df, block_size)
     25             self.examples = []
     26             for _, row in df.iterrows():
---> 27                 conv = construct_conv(row, tokenizer)
     28                 self.examples.append(conv)
     29 

<ipython-input-14-a654172287f5> in construct_conv(row, tokenizer, eos)
      2 def construct_conv(row, tokenizer, eos = True):
      3     flatten = lambda l: [item for sublist in l for item in sublist]
----> 4     conv = list(reversed([tokenizer.encode(x) + [tokenizer.eos_token_id] for x in row]))
      5     conv = flatten(conv)
      6     return conv

<ipython-input-14-a654172287f5> in <listcomp>(.0)
      2 def construct_conv(row, tokenizer, eos = True):
      3     flatten = lambda l: [item for sublist in l for item in sublist]
----> 4     conv = list(reversed([tokenizer.encode(x) + [tokenizer.eos_token_id] for x in row]))
      5     conv = flatten(conv)
      6     return conv

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in encode(self, text, text_pair, add_special_tokens, padding, truncation, max_length, stride, return_tensors, **kwargs)
   2146             stride=stride,
   2147             return_tensors=return_tensors,
-> 2148             **kwargs,
   2149         )
   2150 

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in encode_plus(self, text, text_pair, add_special_tokens, padding, truncation, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
   2474             return_length=return_length,
   2475             verbose=verbose,
-> 2476             **kwargs,
   2477         )
   2478 

/usr/local/lib/python3.7/dist-packages/transformers/models/gpt2/tokenization_gpt2_fast.py in _encode_plus(self, *args, **kwargs)
    171         )
    172 
--> 173         return super()._encode_plus(*args, **kwargs)
    174 
    175     def save_vocabulary(self, save_directory: str, filename_prefix: Optional[str] = None) -> Tuple[str]:

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_fast.py in _encode_plus(self, text, text_pair, add_special_tokens, padding_strategy, truncation_strategy, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
    493             return_length=return_length,
    494             verbose=verbose,
--> 495             **kwargs,
    496         )
    497 

/usr/local/lib/python3.7/dist-packages/transformers/models/gpt2/tokenization_gpt2_fast.py in _batch_encode_plus(self, *args, **kwargs)
    161         )
    162 
--> 163         return super()._batch_encode_plus(*args, **kwargs)
    164 
    165     def _encode_plus(self, *args, **kwargs) -> BatchEncoding:

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_fast.py in _batch_encode_plus(self, batch_text_or_text_pairs, add_special_tokens, padding_strategy, truncation_strategy, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose)
    406             batch_text_or_text_pairs,
    407             add_special_tokens=add_special_tokens,
--> 408             is_pretokenized=is_split_into_words,
    409         )
    410 

TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]]

In the newer versions of discord.py if you run the current script it triggers an error in clients module. It is due to "intents" method. The error is hard to detect and catch for new programmers on discord.py library. I have fixed it and now the code runs perfectly without any errors.

When running the main function, I get the error above.

01/27/2022 19:54:30 - WARNING - __main__ -   Process rank: -1, device: cuda, n_gpu: 1, distributed training: False, 16-bits training: False
/usr/local/lib/python3.7/dist-packages/transformers/models/auto/modeling_auto.py:807: FutureWarning: The class `AutoModelWithLMHead` is deprecated and will be removed in a future version. Please use `AutoModelForCausalLM` for causal language models, `AutoModelForMaskedLM` for masked language models and `AutoModelForSeq2SeqLM` for encoder-decoder models.
  FutureWarning,
01/27/2022 19:54:37 - INFO - __main__ -   Training/evaluation parameters <__main__.Args object at 0x7f71d694fad0>
01/27/2022 19:54:37 - INFO - __main__ -   Creating features from dataset file at cached

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-19-523c0d2a27d3> in <module>()
----> 1 main(trn_df, val_df)

10 frames

<ipython-input-18-aa20b6fc78bc> in main(df_trn, df_val)
     61     # Training
     62     if args.do_train:
---> 63         train_dataset = load_and_cache_examples(args, tokenizer, df_trn, df_val, evaluate=False)
     64 
     65         global_step, tr_loss = train(args, train_dataset, model, tokenizer)

<ipython-input-13-67f62bb60333> in load_and_cache_examples(args, tokenizer, df_trn, df_val, evaluate)
      2 
      3 def load_and_cache_examples(args, tokenizer, df_trn, df_val, evaluate=False):
----> 4     return ConversationDataset(tokenizer, args, df_val if evaluate else df_trn)
      5 
      6 

<ipython-input-12-a654172287f5> in __init__(self, tokenizer, args, df, block_size)
     25             self.examples = []
     26             for _, row in df.iterrows():
---> 27                 conv = construct_conv(row, tokenizer)
     28                 self.examples.append(conv)
     29 

<ipython-input-12-a654172287f5> in construct_conv(row, tokenizer, eos)
      2 def construct_conv(row, tokenizer, eos = True):
      3     flatten = lambda l: [item for sublist in l for item in sublist]
----> 4     conv = list(reversed([tokenizer.encode(x) + [tokenizer.eos_token_id] for x in row]))
      5     conv = flatten(conv)
      6     return conv

<ipython-input-12-a654172287f5> in <listcomp>(.0)
      2 def construct_conv(row, tokenizer, eos = True):
      3     flatten = lambda l: [item for sublist in l for item in sublist]
----> 4     conv = list(reversed([tokenizer.encode(x) + [tokenizer.eos_token_id] for x in row]))
      5     conv = flatten(conv)
      6     return conv

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in encode(self, text, text_pair, add_special_tokens, padding, truncation, max_length, stride, return_tensors, **kwargs)
   2213             stride=stride,
   2214             return_tensors=return_tensors,
-> 2215             **kwargs,
   2216         )
   2217 

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in encode_plus(self, text, text_pair, add_special_tokens, padding, truncation, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
   2552             return_length=return_length,
   2553             verbose=verbose,
-> 2554             **kwargs,
   2555         )
   2556 

/usr/local/lib/python3.7/dist-packages/transformers/models/gpt2/tokenization_gpt2_fast.py in _encode_plus(self, *args, **kwargs)
    172         )
    173 
--> 174         return super()._encode_plus(*args, **kwargs)
    175 
    176     def save_vocabulary(self, save_directory: str, filename_prefix: Optional[str] = None) -> Tuple[str]:

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_fast.py in _encode_plus(self, text, text_pair, add_special_tokens, padding_strategy, truncation_strategy, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
    512             return_length=return_length,
    513             verbose=verbose,
--> 514             **kwargs,
    515         )
    516 

/usr/local/lib/python3.7/dist-packages/transformers/models/gpt2/tokenization_gpt2_fast.py in _batch_encode_plus(self, *args, **kwargs)
    162         )
    163 
--> 164         return super()._batch_encode_plus(*args, **kwargs)
    165 
    166     def _encode_plus(self, *args, **kwargs) -> BatchEncoding:

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_fast.py in _batch_encode_plus(self, batch_text_or_text_pairs, add_special_tokens, padding_strategy, truncation_strategy, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose)
    425             batch_text_or_text_pairs,
    426             add_special_tokens=add_special_tokens,
--> 427             is_pretokenized=is_split_into_words,
    428         )
    429 

TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]]

My Colaboratory Notebook

I've done everything the video was showing except the first two cells in the section "Get Data from Kaggle", because I didn't use kaggle but a json file I scraped from wikiquote of Fullmetal Alchemist, I also made sure not to get any empty values.

But, when I run the main function cell, it throws a TypeError.

Here's the stacktrace:

09/22/2021 23:21:33 - WARNING - __main__ -   Process rank: -1, device: cuda, n_gpu: 1, distributed training: False, 16-bits training: False
/usr/local/lib/python3.7/dist-packages/transformers/models/auto/modeling_auto.py:592: FutureWarning: The class `AutoModelWithLMHead` is deprecated and will be removed in a future version. Please use `AutoModelForCausalLM` for causal language models, `AutoModelForMaskedLM` for masked language models and `AutoModelForSeq2SeqLM` for encoder-decoder models.
  FutureWarning,
09/22/2021 23:21:41 - INFO - __main__ -   Training/evaluation parameters <__main__.Args object at 0x7f0379708d10>
09/22/2021 23:21:41 - INFO - __main__ -   Creating features from dataset file at cached
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-50-523c0d2a27d3> in <module>()
----> 1 main(trn_df, val_df)

10 frames
<ipython-input-49-aa20b6fc78bc> in main(df_trn, df_val)
     61     # Training
     62     if args.do_train:
---> 63         train_dataset = load_and_cache_examples(args, tokenizer, df_trn, df_val, evaluate=False)
     64 
     65         global_step, tr_loss = train(args, train_dataset, model, tokenizer)

<ipython-input-40-67f62bb60333> in load_and_cache_examples(args, tokenizer, df_trn, df_val, evaluate)
      2 
      3 def load_and_cache_examples(args, tokenizer, df_trn, df_val, evaluate=False):
----> 4     return ConversationDataset(tokenizer, args, df_val if evaluate else df_trn)
      5 
      6 

<ipython-input-39-a654172287f5> in __init__(self, tokenizer, args, df, block_size)
     25             self.examples = []
     26             for _, row in df.iterrows():
---> 27                 conv = construct_conv(row, tokenizer)
     28                 self.examples.append(conv)
     29 

<ipython-input-39-a654172287f5> in construct_conv(row, tokenizer, eos)
      2 def construct_conv(row, tokenizer, eos = True):
      3     flatten = lambda l: [item for sublist in l for item in sublist]
----> 4     conv = list(reversed([tokenizer.encode(x) + [tokenizer.eos_token_id] for x in row]))
      5     conv = flatten(conv)
      6     return conv

<ipython-input-39-a654172287f5> in <listcomp>(.0)
      2 def construct_conv(row, tokenizer, eos = True):
      3     flatten = lambda l: [item for sublist in l for item in sublist]
----> 4     conv = list(reversed([tokenizer.encode(x) + [tokenizer.eos_token_id] for x in row]))
      5     conv = flatten(conv)
      6     return conv

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in encode(self, text, text_pair, add_special_tokens, padding, truncation, max_length, stride, return_tensors, **kwargs)
   2160             stride=stride,
   2161             return_tensors=return_tensors,
-> 2162             **kwargs,
   2163         )
   2164 

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in encode_plus(self, text, text_pair, add_special_tokens, padding, truncation, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
   2488             return_length=return_length,
   2489             verbose=verbose,
-> 2490             **kwargs,
   2491         )
   2492 

/usr/local/lib/python3.7/dist-packages/transformers/models/gpt2/tokenization_gpt2_fast.py in _encode_plus(self, *args, **kwargs)
    171         )
    172 
--> 173         return super()._encode_plus(*args, **kwargs)
    174 
    175     def save_vocabulary(self, save_directory: str, filename_prefix: Optional[str] = None) -> Tuple[str]:

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_fast.py in _encode_plus(self, text, text_pair, add_special_tokens, padding_strategy, truncation_strategy, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
    493             return_length=return_length,
    494             verbose=verbose,
--> 495             **kwargs,
    496         )
    497 

/usr/local/lib/python3.7/dist-packages/transformers/models/gpt2/tokenization_gpt2_fast.py in _batch_encode_plus(self, *args, **kwargs)
    161         )
    162 
--> 163         return super()._batch_encode_plus(*args, **kwargs)
    164 
    165     def _encode_plus(self, *args, **kwargs) -> BatchEncoding:

/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_fast.py in _batch_encode_plus(self, batch_text_or_text_pairs, add_special_tokens, padding_strategy, truncation_strategy, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose)
    406             batch_text_or_text_pairs,
    407             add_special_tokens=add_special_tokens,
--> 408             is_pretokenized=is_split_into_words,
    409         )
    410 

TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]]

This is my main method error

NameError                                 Traceback (most recent call last)
<ipython-input-41-523c0d2a27d3> in <module>()
----> 1 main(trn_df, val_df)

NameError: name 'main' is not defined

and this is my method

# Main runner

def main(df_trn, df_val):
    args = Args()
    
    if args.should_continue:
        sorted_checkpoints = _sorted_checkpoints(args)
        if len(sorted_checkpoints) == 0:
            raise ValueError("Used --should_continue but no checkpoint was found in --output_dir.")
        else:
            args.model_name_or_path = sorted_checkpoints[-1]

    if (
        os.path.exists(args.output_dir)
        and os.listdir(args.output_dir)
        and args.do_train
        and not args.overwrite_output_dir
        and not args.should_continue
    ):
        raise ValueError(
            "Output directory ({}) already exists and is not empty. Use --overwrite_output_dir to overcome.".format(
                args.output_dir
            )
        )

    # Setup CUDA, GPU & distributed training
    device = torch.device("cuda")
    args.n_gpu = torch.cuda.device_count()
    args.device = device

    # Setup logging
    logging.basicConfig(
        format="%(asctime)s - %(levelname)s - %(name)s -   %(message)s",
        datefmt="%m/%d/%Y %H:%M:%S",
        level=logging.INFO if args.local_rank in [-1, 0] else logging.WARN,
    )
    logger.warning(
        "Process rank: %s, device: %s, n_gpu: %s, distributed training: %s, 16-bits training: %s",
        args.local_rank,
        device,
        args.n_gpu,
        bool(args.local_rank != -1),
        args.fp16,
    )

    # Set seed
    set_seed(args)

    config = AutoConfig.from_pretrained(args.config_name, cache_dir=args.cache_dir)
    tokenizer = AutoTokenizer.from_pretrained(args.tokenizer_name, cache_dir=args.cache_dir)
    model = AutoModelWithLMHead.from_pretrained(
        args.model_name_or_path,
        from_tf=False,
        config=config,
        cache_dir=args.cache_dir,
    )
    model.to(args.device)
    
    logger.info("Training/evaluation parameters %s", args)

    # Training
    if args.do_train:
        train_dataset = load_and_cache_examples(args, tokenizer, df_trn, df_val, evaluate=False)

        global_step, tr_loss = train(args, train_dataset, model, tokenizer)
        logger.info(" global_step = %s, average loss = %s", global_step, tr_loss)

    # Saving best-practices: if you use save_pretrained for the model and tokenizer, you can reload them using from_pretrained()
    if args.do_train:
        # Create output directory if needed
        os.makedirs(args.output_dir, exist_ok=True)

        logger.info("Saving model checkpoint to %s", args.output_dir)
        # Save a trained model, configuration and tokenizer using `save_pretrained()`.
        # They can then be reloaded using `from_pretrained()`
        model_to_save = (
            model.module if hasattr(model, "module") else model
        )  # Take care of distributed/parallel training
        model_to_save.save_pretrained(args.output_dir)
        tokenizer.save_pretrained(args.output_dir)

        # Good practice: save your training arguments together with the trained model
        torch.save(args, os.path.join(args.output_dir, "training_args.bin"))

        # Load a trained model and vocabulary that you have fine-tuned
        model = AutoModelWithLMHead.from_pretrained(args.output_dir)
        tokenizer = AutoTokenizer.from_pretrained(args.output_dir)
        model.to(args.device)

    # Evaluation
    results = {}
    if args.do_eval and args.local_rank in [-1, 0]:
        checkpoints = [args.output_dir]
        if args.eval_all_checkpoints:
            checkpoints = list(
                os.path.dirname(c) for c in sorted(glob.glob(args.output_dir + "/**/" + WEIGHTS_NAME, recursive=True))
            )
            logging.getLogger("transformers.modeling_utils").setLevel(logging.WARN)  # Reduce logging
        logger.info("Evaluate the following checkpoints: %s", checkpoints)
        for checkpoint in checkpoints:
            global_step = checkpoint.split("-")[-1] if len(checkpoints) > 1 else ""
            prefix = checkpoint.split("/")[-1] if checkpoint.find("checkpoint") != -1 else ""

            model = AutoModelWithLMHead.from_pretrained(checkpoint)
            model.to(args.device)
            result = evaluate(args, model, tokenizer, df_trn, df_val, prefix=prefix)
            result = dict((k + "_{}".format(global_step), v) for k, v in result.items())
            results.update(result)

    return results
    ```

I already tried to reduce my batch to 1, and I am looking for help on how to run it better, my GPU has 6 GB of dedicated VRAM, and still is out of memory, maybe I need to reduce my sample?

Hello Maintainer of Github repo RuolinZheng08/twewy-discord-chatbot (@RuolinZheng08 )!

Thank you for your work on RuolinZheng08/twewy-discord-chatbot. This GitHub project is interesting, and we think that it would be a great addition to make this work instantly discoverable & available as an API for all your users, to quickly try and use it in their applications.

On Tiyaro, every model in RuolinZheng08/twewy-discord-chatbot will get its own: Dedicated model card (see https://console.tiyaro.ai/explore/r3dhummingbird-DialoGPT-medium-joshua Model demo (see https://console.tiyaro.ai/explore/r3dhummingbird-DialoGPT-medium-joshua/demo) Unique Inference API (https://api.tiyaro.ai/explore/huggingface/1//r3dhummingbird/DialoGPT-medium-joshua) Sample code snippets and swagger spec for the API

Users will also be able to compare your model with other models of similar types on various parameters using Tiyaro Experiments (https://blog.tiyaro.ai/evaluate-openmmlabs-mmocr-models-using-tiyaro-experiments)

—- I am from Tiyaro.ai (https://tiyaro.ai/). We are working on enabling developers to instantly evaluate, use and customize the world’s best AI. We are constantly working on adding new features to Tiyaro EasyTrain, EasyServe & Experiments, to make the best use of your ML model, and making AI more accessible for anyone.

Sincerely, I-Jong Lin

Traceback (most recent call last): File "main.py", line 82, in main() File "main.py", line 78, in main client = MyClient('DialoGPT-small-RickAndMorty2') File "main.py", line 17, in init super().init() TypeError: init() missing 1 required keyword-only argument: 'intents'

for step, batch in enumerate(epoch_iterator):

            # Skip past any already trained steps if resuming training
            if steps_trained_in_current_epoch > 0:
                steps_trained_in_current_epoch -= 1
                continue

            inputs, labels = (batch, batch)
            if inputs.shape[1] > 1024: continue
            inputs = inputs.to(args.device)
            labels = labels.to(args.device)
            model.train()
            outputs = model(inputs, labels=labels)

Hello, I did everything as shown in the video, but I get an error when I want to access the "transcript.csv" file https://gyazo.com/c59a07d69a758e2d9abe2820113eae0d

https://www.kaggle.com/renjagrotemeyer/prototech-chat-transcipt