Large-scale pretraining for dialogue

Microsoft

Last update: Jan 7, 2023

Related tags

Text Data & NLP machine-learning dialogue text-generation pytorch transformer data-processing text-data gpt-2 dialogpt

Overview

A State-of-the-Art Large-scale Pretrained Response Generation Model (DialoGPT)

This repository contains the source code and trained model for a large-scale pretrained dialogue response generation model. The human evaluation results indicate that the response generated from DialoGPT is comparable to human response quality under a single-turn conversation Turing test.

The repository is based on huggingface pytorch-transformer and OpenAI GPT-2, containing data extraction script, model training code and pretrained small (117M) medium (345M) and large (762M) model checkpoint.

The model is trained on 147M multi-turn dialogue from Reddit discussion thread. The largest model can be trained in several hours on a 8 V100 machines (however this is not required), with distributed training and FP16 option.

The include script can be used to reproduce the results of DSTC-7 grounded dialogue generation challenge and a 6k multi-reference dataset created from Reddit data.

Project webpage: https://www.microsoft.com/en-us/research/project/large-scale-pretraining-for-response-generation/

ArXiv paper: https://arxiv.org/abs/1911.00536

News

(Update 06/23/2021) We have released a retrieval-augmented/grounded version of DialoGPT (RetGen), please check out the RetGen repo and RetGen paper

(Update 05/20/2021) An awesome video walkthrough on YouTube for DialoGPT by Prakhar Mishra

(Update 03/31/2021) A 3rd party demo by AK391 using Gradio web demo try it out

(Update 09/15/2020) A set of large-scale dialog ranking models has been released!

DialoGPT generation is improved by integrating with our latest dialog ranking models, DialogRPT

(Update 07/08/2020) The 6K multi-ref test set has been released!

To generate the data, pleaser run demo.py and set the data option to 'full', the generated 6k multi-ref test set will be located at

./data/test.refs.txt

(Update 03/10/2020) Model cards available in Huggingface Transformers!

Please check out our model cards in huggingface Transformers repository. With several lines of code it should be pretty straighforward to play with the DialoGPT interactively.

small model: https://huggingface.co/microsoft/DialoGPT-small

medium model: https://huggingface.co/microsoft/DialoGPT-medium

large model: https://huggingface.co/microsoft/DialoGPT-large

(New) Ranking model: https://huggingface.co/microsoft/DialogRPT-updown

(Update 01/06/2020) Some third-party decoding script implementations:

https://github.com/polakowo/gpt2bot GPT2Bot implementation based on telegram by polakowo, ref
https://colab.research.google.com/drive/1PslHE4Rl4RqSa20s7HEp0ZKITBir6ezE A colab interactive notebook by qywu,ref
https://github.com/andreamad8/DialoGPT2-Interact An interactive script featuring multiturn chatbot by andreamad8,ref
https://github.com/LHolten/DialoGTP-MMI-decoder An MMI implementation by LHolten,ref
https://colab.research.google.com/drive/1-_KjlAV3J1IVDw_9KogjKDCzgFY7Jp7E A colab interactive notebook by illuminascent@Reddit,ref
https://colab.research.google.com/drive/15wa925dj7jvdvrz8_z3vU7btqAFQLVlG A great tutorial of how to finetune DialoGPT to build a customized bot built by Rostyslav Neskorozhenyi. ref
https://gradio.app/g/AK391/DialoGPT A 3rd party demo by AK391 using Gradio web demo

Recommended Configuration

Linux Ubuntu 16.04
GPU with at least 12G memory

DialoGPT was developed entirely on Ubuntu 16.04, and -- depending on our availability -- we try to provide support if you experience difficulties running the code on the same configuration. However, we are unable to provide support for other distributions or operating systems. Portions of the code may run on other UNIX flavors (macOS, Windows subsystem for Linux, Cygwin, etc.), but it is recommended to use Ubuntu for the main training code.

The training code can be run on CPU, but it can be slow. We would recommend to use GPU to train and finetune all models. There is no minimal limit of the number of GPUs. However, if using distributed train for multiple GPUs configuration, the speed-up vs the number of GPUs is roughly sub-linear. To simulate the same batchsize when using less GPUs, please use a larger gradient_accumulation_steps in model training.

The 117M and 345M model can be loaded in a single GPU with 12G memory. The 762M model would require a single GPU that has greater than 16G memory for efficient training. The training speed on a benchmark data with 50M training instances and V100 GPUs:

n_gpu	epoch time (h)	token/sec
1	118	10847
2	62	20645
4	34	37647
8	18	71356

Fine-tuning from our pretrained model on a new dataset typically requires 1-2 epochs.

Setup & Installation (TL;DR)

We created a demo script demo.py to ease the difficulty of the deployment of this system. The demo.py contains a pipeline of model downloading, data extraction, data preprocessing and model training over a dummy dataset within one commandline.

Train model with Conda Environment

Please use the below commandlines to clone, install the requirements and load the Conda environment (Note that the Nvidia CUDA 10.0 developer toolkit is required):

sudo apt-get install -y make wget gzip bzip2 xz-utils zstd sed

git clone https://github.com/microsoft/DialoGPT.git
cd DialoGPT
conda env create -f LSP-linux.yml -n LSP
conda activate LSP

If you run this on an architecture other than Linux, please use LSP-generic.yml instead of LSP-linux.yml but please note that the generic one is not tested in all platform, so the stablity can not be gauranteed. To use fp16 training, please install apex by using commands below

conda activate LSP
git clone https://github.com/NVIDIA/apex
cd apex
git reset --hard 3d01e4a0a188cc8df54bc6e44cf5eb40ff6b4cc5
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" .
python3.6 demo.py

Train model with Docker environment

To start, first install the docker and Nvidia-docker from their official repos. The image environment for running the code can be loaded as below:

Nvidia-docker v2.*

$ docker run --gpus all --ipc=host --rm -it -v $PWD:/workspace --network=host icaruszyz/large-scale-training:dialogpt bash

Nvidia-docker v1.*

$ nvidia-docker --rm -it -v $PWD:/workspace --network=host icaruszyz/large-scale-training:dialogpt bash

Inside the docker container, run

python demo.py

Pipeline details

This section explains all components in the demo.py.

Data loading

Before running demo.py, you can set DATA_FOLDER (default value ./models) in demo.py as the place you want to download all the data and pretrained/fine-tuned models. Then simply run

python demo.py

automatically download models and data,
prepare raw data into db that is ready to use for the program,
generate a training scripts.

Note that by default the demo.py will use a dummy data, please specify the Reddit training data by using option --data. Three options are available:dummy,small and full.

python demo.py --data small
python demo.py --data full

The small Reddit data is around 140MB and the full Reddit data is more than 27GB. You can prepare a cup of coffee when processing with the full Reddit data because it takes a long time!

To generate the 6k multi-ref test set data, pleaser run demo.py and set the data option to 'full', the generation will be located at

./data/test.refs.txt

Pretrained model

The pretrained and fine-tuned models are available on azure blobstorage. Please run/see demo.py for more details about how to download/use those models. Or you could download directly by using the links in demo_utils.py.

Preparing data

First, use the prepare4db.sh to convert a tsv data file into the correct format that the following script can recognize. The trainig data need to be then processed into a database file with below commandline:

python prepro.py --corpus $DATA_PATH

Using the training script

The training script can be used in single GPU or multiple GPU settings (distributed training across multiple GPUs within a single node):

python ./LSP_train.py  # Single GPU training
python -m torch.distributed.launch --nproc_per_node=8 ./LSP_train.py  # Training on 8 GPUs

The training script accept several arguments to tweak the training:

Argument	Type	Default value	Description
max_seq_length	`int`	`128`	Maximum number of tokens for each training instance.
train_input_file	`str`	`""`	Path of the training dataset in a .db format
eval_input_file	`str`	`""`	Path of the validation set in a tsv format
continue_from	`int`	`0`	Resuming the training after a specified number of steps
fp16	`boolean`	`True`	Whether to use 16-bits floating point for model training.
train_batch_size	`int`	`4`	Batch size for training
valid_batch_size	`int`	`4`	Batch size for validation
gradient_accumulation_steps	`int`	`2`	Accumulate gradients on several steps
learning_rate	`float`	`1e-5`	Learning rate
lr_schedule	`str`	`noam`	Learning rate schedule can be chosen from [`noam`, `noamwd`, `BERT`, `None`]
num_optim_steps	`int`	`1000000`	Number of training optimization steps
no_token_id	`boolean`	`True`	If set True, using all-zeros token-type embedding.

During the training, two log files will be updated. The train_log.txt and eval_log.txt contains the model loss, perplexity and training speed (tokens/sec) statistics for the training and dev set.

The log file and saved model checkpoint can be found in ./models/output_model

Model decoding

We note that even with properly filtered Reddit dataset, sometimes our model can still generate moderately toxic/inappropriate responses. Due to this reason, we are unable to provide the decoding script at this time (The live demo and decoding script access is upon invitation only now ). We are currently still working on a controlled decoding method to prevent this system from toxic generation. Please stay tuned.

See issues #3 and Reddit discussions for some discussions on third-party decoding methods.

See below for some third-party decoding methods:

https://github.com/polakowo/gpt2bot GPT2Bot implementation based on telegram by polakowo, ref
https://colab.research.google.com/drive/1PslHE4Rl4RqSa20s7HEp0ZKITBir6ezE A colab interactive notebook by qywu,ref
https://github.com/andreamad8/DialoGPT2-Interact An interactive script featuring multiturn chatbot by andreamad8,ref
https://github.com/LHolten/DialoGTP-MMI-decoder An MMI implementation by LHolten,ref
https://colab.research.google.com/drive/1-_KjlAV3J1IVDw_9KogjKDCzgFY7Jp7E A colab interactive notebook by illuminascent@Reddit,ref
https://gradio.app/g/AK391/DialoGPT A 3rd party demo by AK391 using Gradio web demo

Models

We release 6 fine-tuned models which can be further fine-tuned on low-resource user-customized dataset. The total parameters in these models range from 117M to 762M, in accord with OpenAI GPT-2 model sizes.

Model	Fine-tuned from GPT-2	Trained from scratch
DialoGPT 762M model	[link] [huggingface model card]	[link]
DialoGPT 345M model	[link] [huggingface model card]	[link]
DialoGPT 117M model	[link] [huggingface model card]	[link]
DialoGPT 345M model (reverse, for MMI)	link	-
DialogRPT (new ranking models)	link	-

The model files can be loaded exactly as the GPT-2 model checkpoints from Huggingface's Transformers. You can find the corresponding configuration files (merges.txt, config.json, vocab.json) in DialoGPT's repo in ./configs/*.

The reverse model is predicting the source from the target. This model is used for MMI reranking.

The DialogRPT models our recently proposed ranking models used to predict the human feedback (upvotes, replies) of the responses. These models can be used to improve the DialoGPT generation quality (see our EMNLP paper for details).

Retraining full models

Preparation

The first step to retrain the full models is to generate the aforementioned 27GB Reddit dataset. This involves downloading full Reddit submission and comments dumps from https://files.pushshift.io/reddit and creating intermediate files, which overall require 700GB of local disk space. Downloading and processing the full data requires about 1-2 days, depending on your (CPU) compute capabilties (e.g., ~24 hours with 8 cores on a recent computer). Assuming you ran the above setup and installation steps (conda activate LSP, etc.), you can create the full dataset by running either:

python demo.py --data full

cd reddit_extractor; SIZE=full make -j 8; cd ..

The former command calls the latter, so the two methods are equivalent. We recommend the former, as the latter is mostly useful if you run into any problem or want to customize any arguments (e.g., the make command lets you build only a subset of the data). Note that the downloading phase can be error prone, for example based on your geolocation (firewall, etc.). If the above commands fail to generate data/train.tsv, or if that file is not anywhere close to 27GB, it means something went wrong. In that case, you may want to inspect reddit_extractor/wget-log and reddit_extractor/logs/*.log for any obvious error (e.g., wget unable to download from pushshift.io). If error messages don't make sense to you, feel free to contact us. If so, please be sure to include any error messages gathered from these log files.

Training data statistics: the generated training tsv file should be roughly 26.8 GB uncompressed, with 146.8M training instances, 3.87B source tokens, and 2.14B target tokens (including utterance-level 0/1 weights).

Training

We recommand generating the above data using the demo.py --data full, as it (1) generates the data, (2) converts it into DB format, and (3) trains a model using python LSP_train.py. Please directly edit demo.py if you want to customize any of the hyperparameters.

Evaluations

DSTC-7 challenge

Our model achieved the state-of-the-art results in DSTC-7 Challenge response generation task.

Experiment	NIST2	NIST4	BLEU2	BLEU4	METEOR	ENT-4	DIST-1	DIST-2	Avg. Len
Human response	2.62	2.65	12.35%	3.13%	8.31%	10.45	16.66%	67.01%	18.8
DSTC-7 Winner	2.51	2.52	14.35%	1.83%	8.07%	9.03	10.89%	32.49%	15.1
DialoGPT 345M	2.80	2.82	14.16%	2.31%	8.51%	10.08	9.13%	39.73%	16.9
DialoGPT 345M (BS)	2.92	2.97	19.18%	6.05%	9.29%	9.57	15.73%	51.03%	14.2

where ENT represents the Entropy score, and DIST represents the Distinct score. For all metrics except the average length, larger are better.

Note that the superior automatic evaluation comparing to human responses does not necessary imply that our model achieves human parity. Please check out our paper for more detailed analysis.

To fine-tune the 345M DialoGPT model on the DSTC-7 challenge data on a server with 8 V100 GPUs, please run the following commandline (The DSTC data can be found at DSTC-7 repo):

python3 -m torch.distributed.launch --nproc_per_node=8 train_LSP.py --init_checkpoint ./models/medium/medium_ft.pkl --train_input_file ./data/DSTC_train.db --eval_input_file ./data/DSTC_valid.tsv --model_name_or_path ./model/medium/ --learning_rate 1e-4  --train_batch_size 64 --eval_batch_size 64 --no_token_id

The trained model can be found at DSTC medium model

Evaluation

Please downloads the following 3rd-party packages and save into the empty folder 3rdparty:
- mteval-v14c.pl to compute NIST. You may need to install the following perl modules (e.g. by cpan install): XML:Twig, Sort:Naturally and String:Util.
- meteor-1.5 to compute METEOR. It requires Java.
Please follow the DSTC-7 official repo to extract the data, and put data-official-test/test.refs.txt into ./dstc/data/ folder.
Run the extraction script below to produce the human response hypothesis file human.resp.txt:
```
python extract_human.py
```
Finally, to reproduce the results of human hypothesis on DSTC dataset, please run following commands under the repo folder:
```
python batch_eval.py
```

The evaluation results will be generated in the folder ./dstc/eval/

6K multi-ref dataset result

Automatic evaluation

We test on 6K multi-ref dataset from Reddit. The results are summarized in below

Experiment	NIST2	NIST4	BLEU2	BLEU4	METEOR	ENT-4	DIST-1	DIST-2	Avg. Len
Human response	3.41	4.25	17.90%	7.48%	10.64%	11	14.50%	63.00%	13.1
DialoGPT 117M	2.39	2.41	10.54%	1.55%	7.53%	10.78	8.60%	39.90%	12.8
DialoGPT 345M	3	3.06	16.96%	4.56%	9.81%	9.13	6.80%	26.30%	12.2
DialoGPT 762M	2.84	2.9	18.66%	5.25%	9.66%	9.72	7.76%	29.93%	11.2
DialoGPT 345M (BS)	3.4	3.5	21.76%	7.92%	10.74%	10.48	12.38%	48.74%	11.3
DialoGPT 345M (w/MMI)	3.28	3.33	15.68%	3.94%	11.23%	11.25	9.39%	45.55%	17.2

Human evaluation

We further conduct human evaluations (6K examples for each methods, each example is evaluated by 3 human judges). The results show a strong evidence that our generation quality is towards approaching the quality of real human responses, under this non-interactive Turing test:

Relevance: A and B, which one is more relevant to the source prompt.

System A	A Wins (%)	Ties (%)	B Wins (%)	System B
DialoGPT 345M	2671 (45%)	513 (9%)	2816 (47%)	Human responses
DialoGPT 345M	3281 (72%)	394 (9%)	882 (19%)	PersonalityChat
DialoGPT 345M w/ MMI	2871 (48%)	522 (9%)	2607 (43%)	Human responses

Informativeness: A and B, which one is more contentful and informative.

System A	A Wins (%)	Ties (%)	B Wins (%)	System B
DialoGPT 345M	2722 (45%)	234 (4%)	3044 (51%)	Human responses
DialoGPT 345M	3490 (77%)	206 (5%)	861 (19%)	PersonalityChat
DialoGPT 345M w/ MMI	3011 (50%)	234 (4%)	2755 (46%)	Human responses

Human-Like: A and B, which one do you think is more likely to be generated by Human.

System A	A Wins (%)	Ties (%)	B Wins (%)	System B
DialoGPT 345M	2716 (45%)	263 (4%)	3021 (50%)	Human responses
DialoGPT 345M	3462 (76%)	196 (4%)	899 (20%)	PersonalityChat
DialoGPT 345M w/ MMI	2978 (50%)	241 (4%)	2781 (46%)	Human responses

Please see full details in our arxiv paper.

Generated Examples

Multi-turn generation examples from an interactive environment:

Role	Response
User	Does money buy happiness?
Bot	Depends how much money you spend on it .
User	What is the best way to buy happiness ?
Bot	You just have to be a millionaire by your early 20s, then you can be happy .
User	This is so difficult !
Bot	You have no idea how hard it is to be a millionaire and happy . There is a reason the rich have a lot of money

Multi-turn self-playing dialogue with user prompt

Role	Response
User	what is the meaning of life ?
Bot	The meaning is to be with your family and friends .
Bot	I’m going to guess : It means that your parents and friends have loved you . That’s the meaning , I believe .
Bot	I think the meaning is to make sure you’re alive when you die . So you’ll be with your family , friends , and yourself .
Bot	It’s not , but thanks for trying to help .

Commonsense question answering examples

User	Bot
who is the first president of the United States	George Washington
what is the boiling point of water?	I think it's about 212 F.
which one is bigger, sun or moon?	The sun .
which animal has black and white stripes?	A black and white striped cat .

Handling context that does not appear in training examples (note that training examples are sampled from 2005 to 2017)

User	Bot
The trading war between China and US is still happening .	I'm sure the Chinese government and the U . S . will be fine with that as long as they keep their arms on the table and the money flowing .
who won the world cup in 2018 ?	Germany , I believe .
Nvidia's Titan RTX is really good .	I have this card and i love it . No complaints .
Can Porsche beat Tesla with its new Taycan EV ?	I don't think so . The Tesla is more powerful than the porsche .

Our bot's response to philosophy questions

User	Bot
What is the meaning of a good life ?	I think it's that we're all connected to our past lives and the meaning of life is to live the way you want to and how you want to .
Which is more important -- being successful or being happy ?	Being successful is just endless . Being happy is the only thing that matters .
Is peace the only way to stop war ?	Not if the world is not willing to work with you . Peace is the ultimate goal , but peace requires more than that .
How much freedom should people have ?	The right amount of freedom is the freedom to do as you please , as long as you don't hurt people or property .

Finally

User	Bot
What do you think of the DialoGPT repo ?	It's pretty good , but it's very easy to find a bug .

Please start a issue if you spot any :)

Try our system

The live demo and decoding script access is upon invitation only now. Please stayed tuned for the full release.

Related Project

RetGen: https://github.com/dreasysnail/RetGen. Retrieval-augmented/grounded DialoGPT and beyond. RetGen is a joint training framework that simultaneously optimizes a dense passage retriever and a knowledge-grounded text generator in an end-to-end fashion.
Microsoft ICECAPS: https://github.com/microsoft/icecaps.

As an orthogonal repository of this project, Microsoft Icecaps is an open-source toolkit (in tensorflow) for building neural conversational systems. Icecaps provides an array of tools from recent conversation modeling and general NLP literature within a flexible paradigm that enables complex multi-task learning setups.
Pretrained UniLM: https://github.com/microsoft/unilm
MT-DNN: https://github.com/namisan/mt-dnn
A chinese counterpart of DialoGPT by yangjianxin1. https://github.com/yangjianxin1/GPT2-chitchat. We are glad to see that the MMI strategy that we used in DialoGPT has also improved the performance for this project as well!

Contact

Please contact [email protected] if you have any questions/suggestions. However, the response will be sporadic. Please expect delay.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Disclaimer

This repository aims to facilitate research in large-scale pretraining for conversational data. This toolkit contains only part of the modeling machinery needed to actually produce a model weight file in a running dialog. On its own, this model provides only information about the weights of various text spans; in order for a researcher to actually use it, they will need to bring conversational data of their own and decode the response generation from the pretrained system. Microsoft is not responsible for any generation from the 3rd party utilization of the pretrained system.

Citation

If you use this code in your research, you can cite our arxiv paper:

@inproceedings{zhang2019dialogpt,
    title={DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation},
    author={Yizhe Zhang and Siqi Sun and Michel Galley and Yen-Chun Chen and Chris Brockett and Xiang Gao and Jianfeng Gao and Jingjing Liu and Bill Dolan},
    year={2020},
    booktitle={ACL, system demonstration}
}

Comments

Large model (762M) keeps repeating the same utterance

I tried running the large model (in a colab notebook) using the approach described in the model card from the huggingface library:

from transformers import AutoModelWithLMHead, AutoTokenizer
import torch


tokenizer = AutoTokenizer.from_pretrained("microsoft/DialoGPT-large")
model = AutoModelWithLMHead.from_pretrained("microsoft/DialoGPT-large")

# Let's chat for 5 lines
for step in range(5):
    # encode the new user input, add the eos_token and return a tensor in Pytorch
    new_user_input_ids = tokenizer.encode(input(">> User:") + tokenizer.eos_token, return_tensors='pt')

    # append the new user input tokens to the chat history
    bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids

    # generated a response while limiting the total chat history to 1000 tokens, 
    chat_history_ids = model.generate(bot_input_ids, max_length=1000, pad_token_id=tokenizer.eos_token_id)

    # pretty print last ouput tokens from bot
    print("DialoGPT: {}".format(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True)))

Unfortunately, the output seems to quickly repeat the same sentence over and over again. In example 1 and 2, it repeats the same sentence from the beginning. In the 3rd case, the model starts fine, but as the conversation progresses, it starts repeating the same thing (or parts of it).

Is that an intended behavior?

Example 1

>> User:Today, I bought some ice cream.
DialoGPT: I'm so sorry.
>> User:Why are you sorry?
DialoGPT: I'm sorry for your loss.
>> User:What loss? I didn't lose anything.
DialoGPT: I'm sorry for your loss.
>> User:I'm very confused by what you are saying.
DialoGPT: I'm sorry for your loss.
>> User:Ok, nevermind.
DialoGPT: I'm sorry for your loss.

Example 2

>> User:Can't wait to upvote this 17 different times later this week.
DialoGPT: I'm going to upvote it every time.
>> User:What are you going to upvote every time?
DialoGPT: I'm going to upvote it every time.
>> User:Uh sure, if that's what you want.
DialoGPT: I'm going to upvote it every time.
>> User:I'm very confused by what you are saying.
DialoGPT: I'm going to upvote it every time.
>> User:What's your name?
DialoGPT: I'm going to upvote it every time.

Example 3

>> User:I like to walk my dog in the park.
DialoGPT: I like to walk my dog in the park.
>> User:That's great. What's your dog breed?
DialoGPT: He's a mix of some sort.
>> User:That's funny, you don't even know what mix it is.
DialoGPT: I know what mix it is.
>> User:What is it, then?
DialoGPT: A mix of some sort.
>> User:What sort? "Sort" is not a type of dog.
DialoGPT: I know what mix it is.
>> User:That's not the issue. I'm talking about your dog's breed.
DialoGPT: I know what mix it is.
>> User:Then tell me!
DialoGPT: I know what mix it is.
>> User:What are you waiting for?
DialoGPT: I know what mix?
>> User:Are you serious?
DialoGPT: I know what?
>> User:What?
DialoGPT: I know what?

opened by xhluca 10

multi-turn dialog format

Section 3.1 of the paper states that dialog turns of the same session are concatenated into a long text, ended by the end-of-text token.

Does this mean that there are no special tokens in between dialog turns?

How do I separate dialog turns?

opened by LHolten 7
Downloading Reddit dataset small - make: wget: No such file or directory

Hi,

When I tried to download the Reddit dataset (small), I encountered the following problem.

$ cd reddit_extractor; make -j 8; cd ..

wget https://files.pushshift.io/reddit/submissions/RS_v2_2005-12.xz -O data/reddit/RS_v2_2005-12.xz -o logs/RS_v2_2005-12.xz.log -c make: wget: No such file or directory wget https://files.pushshift.io/reddit/comments/RC_2005-12.bz2 -O data/reddit/RC_2005-12.bz2 -o logs/RC_2005-12.bz2.log -c make: wget: No such file or directory wget https://files.pushshift.io/reddit/submissions/RS_v2_2006-01.xz -O data/reddit/RS_v2_2006-01.xz -o logs/RS_v2_2006-01.xz.log -c make: wget: No such file or directory wget https://files.pushshift.io/reddit/comments/RC_2006-01.bz2 -O data/reddit/RC_2006-01.bz2 -o logs/RC_2006-01.bz2.log -c make: wget: No such file or directory make: *** [data/reddit/RC_2005-12.bz2] Error 1 make: *** Waiting for unfinished jobs.... make: *** [data/reddit/RS_2005-12.bz2] Error 1 make: *** [data/reddit/RS_2006-01.bz2] Error 1 make: *** [data/reddit/RC_2006-01.bz2] Error 1

I'd like to download the dataset without training the model. Before running the make file, I installed the Conda environment listed in LSP-generic.yml without cudatoolkit=10.0.130=0 on a Mac. I checked reddit_extractor/logs/ but it is empty. Could someone help point out a possible direction to solve this problem? Thanks.

opened by haozhe-an 4
Understanding the train.tsv file
After running python demo.py --data small and looking at the resulting train.tsv file, I want to make sure I have the correct understanding of the format and what the float values indicate.

For example, the first two examples look like:

t3_17830,t1_c24,t1_c40 0.0 On the bright side , despite kidnapping and cruelly abandoning him , it doesn't sound like he was tortured ... 1.0 We didn't torture somebody ! USA t3_17844,t1_c88,t1_c95 1.0 will comments dissapear if ranked low enough ? I can just see the pages with 5000 comments now .. 1.0 not yet , but we'll play around with it

From the paper, I see that there was some heavy preprocessing and filtering done, such as removing offensive and bland training instances. Are the sequences prepended with 0.0 the filtered instances that will not be used to update the weights during training? Based on my understanding of the code, the weight 0.0 ensures this by setting the language modeling labels to -1:

https://github.com/microsoft/DialoGPT/blob/18d91ce5a4e1c32e2b097829c5c3de5135879420/prepro.py#L108-L110

What I'm confused with is that I can't seem to find how the training process ignores the preprended identifiers of each line (ex: t3_17830,t1_c24,t1_c40). How does this part of the training data gets ignored?
opened by wise-east 4
Fix 'no such file(train.tsv.gz) error'

/reddit_extractor/Makefile already expands the %.tsv.gz files in /redditor_extractor/data/out/ and save the contents in /data/ as %.tsv. Therefore, the code in /demo.py to decompress train.tsv.gz is removed.

opened by wang-tc 2
LSP_train.py - Training stuck at 0/num_optimum_steps
Issue : The pbar for training is stuck at 0/num_optim_steps. Could be seen in the image below.

When i clone the repository and run python demo.py everything works find until the training loop. Then the pbar is stuck at 0/num_optim_steps meaning that the training does not take place. By default python demo.py takes --data = dummy_data.

Steps taken towards resolution

Manually ran the command generated such as python LSP_train.py \ --model_name_or_path /home/msoni/DialoGPT/models/small \ --init_checkpoint /home/msoni/DialoGPT/models/small/pytorch_model.bin \ --train_input_file /home/msoni/DialoGPT/data/train.128len.db \ --eval_input_file ./data/dummy_data.tsv \ --output_dir /home/msoni//DialoGPT/models/output_model \ --seed 42 \ --max_seq_length 128 \ --train_batch_size 8 \ --gradient_accumulation_steps 8 \ --eval_batch_size 64 \ --learning_rate 1e-05 \ --num_optim_steps 1000 \ --valid_step 500 \ --warmup_steps 100 \ --normalize_data true \ --fp16 False \ --lr_schedule noam \ --loss_scale 0.0 \ --no_token_id true \ --pbar true \ --local_rank -1

Altered the parameters for num_optim_steps, valid_steps, and warmup_steps.

Turned from_scratch = True

After the while loop at line 273 in LSP_train.py, printed a statement if that line executes. The code skips all lines after line number 290. This is the place the skip starts

if args.no_token_id: token_ids = None loss, ppl = model(input_ids, position_ids, token_ids, label_ids) logger.info("loss", loss) logger.info("ppl", ppl) logger.info("if this command print than line number 294 has run")

Is there something obvious that i am missing ?
opened by Mayanksoni20 2
"fs" vs "ft" model differences

Hi, thanks very much for the repo. I am using the downloaded models: https://github.com/microsoft/DialoGPT/blob/master/demo_utils.py, and want to ask what are the differences between "medium_fs.pkl" vs "medium_ft.pkl", or "small_fs.pkl" vs "small_ft.pkl"? Do they impact the performance and accuracy of the model?

opened by hieucnguyen 2
Can not find a model after training - strange

Hi, cool repo. I fine-tune small_ft.pkl with the following training parameter in Google Colab:

!python LSP_train.py --model_name_or_path /content/DialoGPT/models/small --init_checkpoint /content/apex/DialoGPT/small_ft.pkl --train_input_file /content/apex/DialoGPT/data/train_raw_friend.128len.db --eval_input_file ./data/dummy_data.tsv --output_dir /content/DialoGPT/models/output_model --seed 42 --max_seq_length 128 --train_batch_size 16 --gradient_accumulation_steps 8 --eval_batch_size 64 --learning_rate 1e-5 --num_optim_steps 1000 --valid_step 5000 --warmup_steps 4000 --normalize_data true --fp16 false --lr_schedule noam --loss_scale 0.0 --no_token_id true --pbar true

Trainig finished but when i navigate to /content/DialoGPT/models/output_model there is no actual model ! i see only 2 files in generated folder GPT2.1e-05.2.1gpu.2020-04-04161237 eval_log and train_log, but no large pkl file.

What i am doing wrong?

opened by GraphGrailAi 2
Maximum sequence length

On the README, it says that the default max_seq_length is 128. Was this the actual parameter used for training the models?

If so, does that mean without further fine-tuning or training a model from scratch with a longer max_seq_length, the trained models will not be able to generate proper responses for input sequences that are longer than 128 or have the responses cut short at 128? I'm asking because it seems like the case if I provide a context that is too long, about 120 tokens, then it would only generate about 8 tokens and stop, making the output get cut short.

opened by wise-east 2

Code stuck after logging number of parameters

I ran python demo.py after replacing the dummy_data.tsv with my own small corpus. After reaching the train part the code just froze at this point.

11/06/2019 16:23:01 - INFO - __main__ -   Generating training CMD!
11/06/2019 16:23:01 - INFO - __main__ -   If there is any problem, please copy (modify) and run command below
11/06/2019 16:23:01 - INFO - __main__ -   #########################################################################
python LSP_train.py --model_name_or_path ./models/medium --init_checkpoint ./models/medium/pytorch_model.bin --train_input_file ./data/train.128len.db --eval_input_file ./data/ss_val_data.tsv --output_dir ./models/output_model --seed 42
--max_seq_length 128 --train_batch_size 512 --gradient_accumulation_steps 8 --eval_batch_size 64 --learning_rate 1e-5 --num_optim_steps 47500 --valid_step 475 --warmup_steps 9500 --normalize_data true --fp16 false --lr_schedule noam --loss_scale 0.0 --no_token_id true --pbar true
11/06/2019 16:23:01 - INFO - __main__ -   #########################################################################
11/06/2019 16:23:02 - INFO - __main__ -   train batch size = 512, new train batch size (after gradient accumulation) = 64
11/06/2019 16:23:05 - INFO - __main__ -   CUDA available? True
11/06/2019 16:23:05 - INFO - __main__ -   Input Argument Information
11/06/2019 16:23:05 - INFO - __main__ -   model_name_or_path            ./models/medium
11/06/2019 16:23:05 - INFO - __main__ -   seed                          42
11/06/2019 16:23:05 - INFO - __main__ -   max_seq_length                128
11/06/2019 16:23:05 - INFO - __main__ -   skip_eval                     False
11/06/2019 16:23:05 - INFO - __main__ -   init_checkpoint               ./models/medium/pytorch_model.bin
11/06/2019 16:23:05 - INFO - __main__ -   train_input_file              ./data/train.128len.db
11/06/2019 16:23:05 - INFO - __main__ -   eval_input_file               ./data/ss_val_data.tsv
11/06/2019 16:23:05 - INFO - __main__ -   continue_from                 0
11/06/2019 16:23:05 - INFO - __main__ -   train_batch_size              64
11/06/2019 16:23:05 - INFO - __main__ -   gradient_accumulation_steps   8
11/06/2019 16:23:05 - INFO - __main__ -   eval_batch_size               64
11/06/2019 16:23:05 - INFO - __main__ -   learning_rate                 1e-05
11/06/2019 16:23:05 - INFO - __main__ -   num_optim_steps               47500
11/06/2019 16:23:05 - INFO - __main__ -   valid_step                    475
11/06/2019 16:23:05 - INFO - __main__ -   warmup_proportion             0.1
11/06/2019 16:23:05 - INFO - __main__ -   warmup_steps                  9500
11/06/2019 16:23:05 - INFO - __main__ -   normalize_data                True
11/06/2019 16:23:05 - INFO - __main__ -   fp16                          False
11/06/2019 16:23:05 - INFO - __main__ -   lr_schedule                   noam
11/06/2019 16:23:05 - INFO - __main__ -   loss_scale                    0.0
11/06/2019 16:23:05 - INFO - __main__ -   no_token_id                   True
11/06/2019 16:23:05 - INFO - __main__ -   output_dir                    ./models/output_model
11/06/2019 16:23:05 - INFO - __main__ -   log_dir                       None
11/06/2019 16:23:05 - INFO - __main__ -   pbar                          True
11/06/2019 16:23:05 - INFO - __main__ -   local_rank                    -1
11/06/2019 16:23:05 - INFO - __main__ -   config                        None
11/06/2019 16:23:05 - INFO - __main__ -   device                        cuda
11/06/2019 16:23:05 - INFO - __main__ -   n_gpu                         1
11/06/2019 16:23:05 - INFO - pytorch_pretrained_bert.tokenization_gpt2 -   loading vocabulary file ./models/medium/vocab.json
11/06/2019 16:23:05 - INFO - pytorch_pretrained_bert.tokenization_gpt2 -   loading merges file ./models/medium/merges.txt
11/06/2019 16:23:09 - INFO - gpt2_training.train_utils -   loading finetuned model from ./models/medium/pytorch_model.bin
11/06/2019 16:23:10 - INFO - gpt2_training.train_utils -   loading transfomer only
11/06/2019 16:23:13 - INFO - __main__ -   Number of parameter = 354823168

nothing logged after this point. Am I missing something or doing something wrong here?

opened by abaheti95 2

Facing an issue where the model will generate responses out of order when encountering dialect
I have realised when if I am talking to the chatbot and suddenly use a dialect word or phrase , the entire flow of the conversation become out-of-order. I have no problems with question in Standard English.

For example:

User: How are you? Bot: I'm fine thanks! User: Happy Birthday! Bot: Thanks :D

I will then use the question, "How old yuh is?" and this is what happens:

User: How are you? Bot: I'm fine thanks! User: Happy Birthday! Bot: User: How old yuh is? Bot: I am 21. Thanks :D

If I continue with another normal english question like "What is the time?" I will get this:

User: What is the time? Bot: It's at the top of the page. User: How are you? Bot: I'm fine thanks! User: Happy Birthday! Bot: User: How old yuh is? Bot: I am 21. Thanks :D

So the entire conversation is now confusing because of being out of order. If I use standard English question, the chatbot behaves fine. I am using the standard code provided. Is there anyway to troubleshoot these occurrences?
opened by KestonSmith 1
docs: demo, experiments and live inference API on Tiyaro

Hello DialoGPT team (@mattetti @mtodd @sverrejoh @radical )!

Thank you for your work on DialoGPT. This project is interesting, and we think that it would be a great addition to make this work instantly discoverable & available as an API for all your users, to quickly try and use it in their applications.

On Tiyaro, every model in DialoGPT will get its own: Dedicated model card (e.g., https://console.tiyaro.ai/explore/microsoft-DialoGPT-large) Model demo (e.g. https://console.tiyaro.ai/explore/microsoft-DialoGPT-large/demo) Unique Inference API (e.g. https://api.tiyaro.ai/v1/ent/huggingface/1/microsoft/DialoGPT-large) Sample code snippets and swagger spec for the API

Users will also be able to compare your model with other models of similar types on various parameters using Tiyaro Experiments (e.g. https://console.tiyaro.ai/experiments/HjSM9viavGwMe9fn1BlW?wId=123456789012&runId=123456789012-experiment-51d9be47f6654dbc88bd9f42d2466f34&innerTab=results)

—- I am from Tiyaro.ai (https://tiyaro.ai/). We are working on enabling developers to instantly evaluate, use and customize the world’s best AI. We are constantly working on adding new features to Tiyaro EasyTrain, EasyServe & Experiments, to make the best use of your ML model, and making AI more accessible for anyone.

Sincerely, I-Jong Lin Founding Engineer at Tiyaro

opened by ijonglin 0
Finetune on model other than GPT-2

hello, I would be grateful if someone answer this question clearly: Can dialogpt finetuned on model other than GPT-2, if so, how?. I tried to finetune this model to GPT-J, as I changed the LSP_train.py line 195 from model = load_model(GPT2LMHeadModel(config), args.init_checkpoint, args, verbose=True) to model = load_model(GPTJForCausalLM.from_pretrained('EleutherAI/gpt-j-6B),args.init_checkpoint, args,verbose=True) but get this error: File "LSP_train.py", line 287, in <module> loss, ppl = model(input_ids, position_ids, token_ids, label_ids) File "/opt/conda/envs/dialogpt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/opt/conda/envs/dialogpt/lib/python3.7/site-packages/transformers/models/gptj/modeling_gptj.py", line 832, in forward return_dict=return_dict, File "/opt/conda/envs/dialogpt/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/opt/conda/envs/dialogpt/lib/python3.7/site-packages/transformers/models/gptj/modeling_gptj.py", line 589, in forward past_length = past_key_values[0][0].size(-2) IndexError: dimension specified as -2 but tensor has no dimensions

The script above get an error when I'm using either GPU or CPU, but it's working fine on gpt-2 model. Would appreciate any help!

opened by raihan0824 0

Response in medium model decreases overtime.

I'm using a modified version of the example code provided in the huggingface website.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch


tokenizer = AutoTokenizer.from_pretrained("microsoft/DialoGPT-medium")
model = AutoModelForCausalLM.from_pretrained("microsoft/DialoGPT-medium")


step = 0
while True:

    # encode the new user input, add the eos_token and return a tensor in Pytorch
    new_user_input_ids = tokenizer.encode(input("You: ") + tokenizer.eos_token, return_tensors='pt')

    # append the new user input tokens to the chat history
    bot_input_ids = torch.cat([chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids

    # generated a response while limiting the total chat history to 1000 tokens
    chat_history_ids = model.generate(
        bot_input_ids, 
        pad_token_id=tokenizer.eos_token_id,
        max_length=1000,
        sample=True,
        top_k=50,
        top_p=0.95,
        repetition_penalty=1.35
    )

    # pretty print last ouput tokens from bot
    print(tokenizer.decode(chat_history_ids[:, bot_input_ids.shape[-1]:][0], skip_special_tokens=True))
    step += 1

After some few lines, the responses starts become shorter and shorter until it just doesn't output anything anymore.

You: hello
Hello! :D
You: coffee tastes so good lol
It does. I'm going to have a coffee tomorrow morning, and it's gonna be delicious haha
You: what's your type of coffee?
I don't drink coffee but if you want some I'll make one for ya! It will probably taste like a cup of tea though...
You: Yes please make me one
Sure thing!
You: I prefer light coffee alright?
Yeah that sounds great!
You: to be honest i just drink coffee for the sake of tasting good lol
Haha okay then ill try my best with this one too XD
You: cool
Sounds awesome ltsss
You: do you prefer your coffee dark or light
ooooo
You: what?
yayyy
You: what are you saying?

You:

I tried changing the max_length to about 5000 and it doesn't seem to do anything. I've tried getting rid of the history (i.e., just using the new_user_input_ids variable) and it seemed to have fix the issue but that obviously leads me to a very random-like response as it has no context on what we're talking about

opened by bossauh 5

make file error

Hi, there,

hope you have a good day. When I compile the demo.py --data full, I had makefile error. I'm wondering if anyone could help me to take a look?

`(LSP) oem@Yueqi:~/DialoGPT$ python demo.py --data full Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex. PROJECT_FOLDER = /home/oem/DialoGPT Found existing models folder at /home/oem/DialoGPT/models, skip creating a new one! 01/31/2022 23:02:56 - INFO - main - Downloading models... 01/31/2022 23:02:56 - INFO - demo_utils - /home/oem/DialoGPT/models/small/config.json exists, return! 01/31/2022 23:02:56 - INFO - demo_utils - /home/oem/DialoGPT/models/small/vocab.json exists, return! 01/31/2022 23:02:56 - INFO - demo_utils - /home/oem/DialoGPT/models/small/merges.txt exists, return! 01/31/2022 23:02:56 - INFO - demo_utils - /home/oem/DialoGPT/models/small/pytorch_model.bin exists, return! 01/31/2022 23:02:56 - INFO - demo_utils - /home/oem/DialoGPT/models/small/small_ft.pkl exists, return! 01/31/2022 23:02:56 - INFO - main - Done!

01/31/2022 23:02:56 - INFO - main - Downloading and Extracting Data... make: *** [Makefile:65: data/reddit/RS_2018-08.bz2] Error 8 make: *** Waiting for unfinished jobs.... make: *** [Makefile:65: data/reddit/RS_2018-03.bz2] Error 8 make: *** [Makefile:65: data/reddit/RS_2018-05.bz2] Error 8 make: *** [Makefile:65: data/reddit/RS_2018-06.bz2] Error 8 make: *** [Makefile:65: data/reddit/RS_2018-07.bz2] Error 8 make: *** [Makefile:65: data/reddit/RS_2018-04.bz2] Error 8

`

opened by NicoleLiYueqi 3

Not able to convert the model to Torchscript

I am trying to convert the DialogGPT model to Torchscript to load the model into Triton. I tried to use JIT to trace the model as mentioned here but hitting the following warning

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
tokenizer = AutoTokenizer.from_pretrained("microsoft/DialoGPT-large")
model = AutoModelForCausalLM.from_pretrained(
    "microsoft/DialoGPT-large", torchscript=True)
step = 0
new_user_input_ids = tokenizer.encode(
    "This is a test!" + tokenizer.eos_token, return_tensors='pt')
bot_input_ids = torch.cat(
    [chat_history_ids, new_user_input_ids], dim=-1) if step > 0 else new_user_input_ids
traced_model = torch.jit.trace(
    model, bot_input_ids)
torch.jit.save(traced_model, "DialogGPT.pt")

/opt/conda/lib/python3.8/site-packages/transformers/models/gpt2/modeling_gpt2.py:196: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  attn_weights = attn_weights / (float(value.size(-1)) ** 0.5)

How can I trace with the generate method?

model.generate(bot_input_ids, max_length=1000, pad_token_id=tokenizer.eos_token_id)

I tried to cover the model ONNX format using this instruction but this takes three inputs input_ids, attention_mask and token_type_ids. I am able to get input_ids and attention_mask from the tokenizer. How can I get the token_type_ids

inputs = tokenizer("How are you doing?", return_tensors="np")

opened by arunraman 0