Implementation of PersonaGPT Dialog Model

Overview

PersonaGPT

An open-domain conversational agent with many personalities

PersonaGPT is an open-domain conversational agent cpable of decoding personalized and controlled responses based on user input. It is built on the pretrained DialoGPT-medium model, following the GPT-2 architecture. PersonaGPT is fine-tuned on the Persona-Chat dataset, with added special tokens to better distinguish between conversational history and personality traits for dyadic conversations. Furthermore, some active learning was used to train the model to do controlled decoding based on certain "action codes" (e.g., talk about work, ask about music interests).


Personalized Dialog Generation

Also called persona modeling, personalized dialog generation takes as input a set of personality facts and dialog history to generate responses that are (ideally) sensible and diverse, yet consistent with past responses and the given persona profile.

Example

Persona Facts
1 i am 64 years old but feel quite young.
2 i try to eat healthy but limit mcdonalds to once a week.
3 i regret working as a doctor for the last 20 years.
4 my secret hobby is making self-help youtube videos.
5 i've been to spain many times and wish i could go again.
Role Response
Human yo whats good fam?
Bot hello, how are you doing?
Human not bad what you doing today?
Bot eating a lot healthier now that i'm in my 60s
Human i'm more of a fast food guy
Bot me too but i'm very healthy so i try to avoid it
Human what do you do for fun?
Bot i make videos for fun and eat very healthy
Human nice what kind of videos?
Bot i love making videos about self help stuff
Human i think i need some help
Bot what kind of help would you like to get
Human i'm 21 and feel lost in life
Bot i can relate, i'm only 65

Controlled Response Generation

Another feature of PersonaGPT is generating responses that directs conversations toward certain topics of interests. The following is a self-play episode between 2 PersonaGPT agents, one interviews the other.

Bot 2's Persona Facts
1 i am a quiet engineer.
2 i'm single and am looking for love.
3 sadly, i don't have any relatable hobbies.
4 luckily, however, i am tall and athletic.
5 on friday nights, i watch re-runs of the simpsons alone.
Role Action Response
Bot 1 talk about movies hello do you like movies?
Bot 2 - i love the simpsons. how about you?
Bot 1 talk about music i like the walking dead. do you play an instrument?
Bot 2 - yes i play the violin.
Bot 1 ask about marital status are you married or single?
Bot 2 - i am single but i am looking for love.
Bot 1 talk about work my job is working in a factory. what's your occupation?
Bot 2 - engineer. i'm very quiet so no one hears me.
Bot 1 talk about hobbies do you have any hobbies?
Bot 2 - i watch reruns of the simpsons.
Bot 1 ask about favorite food what's your favorite food?
Bot 2 - i love pizza. how about yourself?
Bot 1 ask about pets i also love pizza. do you like animals?
Bot 2 - i have two dogs. what is your occupation?
Bot 1 talk about work i'm a factory worker. what's your dream job?
Bot 2 - i'd love to be a writer one day.

Interactive Environment

To interact with PersonaGPT, run interact.py --mode 0 to manually enter responses and see how the bot responds. You will first be prompted to enter a set of persona facts for your partner and then engage in a short conversation.

Use -turns or --num_turns to adjust the length of conversation (default = 8). To adjust the decoding parameters of PersonaGPT, use --help to bring up available settings to tune. For example, --top_k (default = 10) controls the number of candidate tokens at each decoding step, and --top_p controls the nucleus sampling. See how to generate text from Huggingface for nuances of the GPT decoding process.

To engage in controlled response generation, run interact.py --mode 1. At each turn, you will be prompted to select from a set of topics from which to steer the conversation toward. The output response by PersonaGPT may not always steer the conversation to the topic of interest, depending on the dialog history up to the current turn.


Training

Below are instructions to reconstruct the PersonaGPT from "scratch" (i.e., from pretrained DialoGPT or GPT-2, either of which are feasible as starting pre-trained models).

Requirements:

  • Python 3.6+
  • Pytorch (GPU preferred)
  • transformers
  • dotenv
  • tqdm
  • (optional) apex for fp16 training It is highly recommended that the pytorch and transformers packages are installed under a virtual environment.

After cloning this repository, follow the directions below to set up the training environment.

Instructions:

  1. Go to the .env file and set the save_path to your desired local repository to store model, scheduler and optimizer checkpoints. Point data_path to the ~/data folder of the cloned repository. The .env file also contains the hyperparameter configurations:
epochs = 3
learn_rate = 5e-5
gradient_accumulation_steps = 64
batch_size = 1
weight_decay = 0.0
logging_steps = 10
save_steps = 250

Replace epochs, batch_size, gradient_accumulation_steps and learn_rate with the desired hyperparameters of choice. Please use batch_size = 1 and change gradient accumulation steps to adjust the training batch size. This current repo version does not support parallel batching at the moment (TODO).

  1. Run preprocess_dataset.py to preprocess ~/data/train_both_original_no_cands.txt and ~/data/valid_both_original_no_cands.txt. The original .txt files are obtained from the ConvAI2 Challenge, which may no longer be available since the ConvAI3 challenge has taken place. The ConvAI2 challenge data uses the Persona-Chat dataset which is what is provided under the ~/data folder.

  2. Run train.py to train the PersonaGPT model. Results (e.g., pretrain_loss, persona_loss, ctrl_loss) will be saved under [save_path]/samples/. Model checkpoints are saved under [save_path]/checkpoint/model.

Currently there are 2 training loops, pretrain() and train_loop(). pretrain() first trains model on the Persona-Chat dataset and saves the performance under pretrain_stats. train_loop() then fine-tunes the model on active learning data, which examples of using action codes (e.g., "talk about hobbies", "ask about pets") to do controlled response generation. The pretrained model can be used as as stand-alone dialog model for personalized dialog generation without fine-tuning on the actively learned actions.

  • pretrain_loss: tracks the training loss on Persona-Chat dataset during pretrain().
  • persona_loss: tracks the training loss on Persona-Chat during train_loop().
  • ctrl_loss: tracks the training loss on actively learned action codes during train_loop().

Active Learning

Currently, there are 11 possible turn-level goals that can be used for controlled response generation.

Turn-level Goals
1. ask about family. 4. talk about traveling. 7. talk about music.
2. ask about pets. 5. ask about age and gender. 8. talk about food.
3. talk about work. 6. talk about hobbies. 9. talk about movies.
10. talk about politics. 11. ask about marital status. -

These turn-level goals are handcrafted based on the personachat dataset to cover most of the conversational directions at the turn-level.

To actively learn new turn-level goals, use the convogym repo.


Evaluation

After training, an evaluation loop will run and print out a set of scores saved under eval_stats. Below is a comparison of PersonaGPT vs. other baselines on the Persona-Chat dataset using automatic evaluation metrics. Your results should look something like:

Model Perplexity F1 Score
Seq2seq Baseline [3] 29.8 16.2
Wolf et al. [5] 16.3 19.5
GPT-2 baseline 99.5 5.8
DialoGPT baseline 56.6 12.6
DialoGPT finetuned 11.4 22.7
PersonaGPT 10.2 43.4

Cite Us

Our full paper is now up on arXiv.

@misc{tang2021persona,
      title={Persona Authentication through Generative Dialogue}, 
      author={Fengyi Tang and Lifan Zeng and Fei Wang and Jiayu Zhou},
      year={2021},
      eprint={2110.12949},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

References

  1. Radford, Alec, et al. "Language models are unsupervised multitask learners." OpenAI Blog 1.8 (2019): 9.

  2. Zhang, Yizhe, et al. "Dialogpt: Large-scale generative pre-training for conversational response generation." arXiv preprint arXiv:1911.00536 (2019).

  3. Zhang, Saizheng, et al. "Personalizing dialogue agents: I have a dog, do you have pets too?." arXiv preprint arXiv:1801.07243 (2018).

  4. Dinan et al., "The Second Conversational Intelligence Challenge (ConvAI2)." arXiv preprint arXiv:1902.00098 (2019).

  5. Thomas Wolf et al. "Transfertransfo: A transfer learning approach for neural network based conversational agents." arXiv preprint328arXiv:1901.08149, 2019

You might also like...
ReConsider is a re-ranking model that re-ranks the top-K (passage, answer-span) predictions of an Open-Domain QA Model like DPR (Karpukhin et al., 2020).

ReConsider ReConsider is a re-ranking model that re-ranks the top-K (passage, answer-span) predictions of an Open-Domain QA Model like DPR (Karpukhin

Ever felt tired after preprocessing the dataset, and not wanting to write any code further to train your model? Ever encountered a situation where you wanted to record the hyperparameters of the trained model and able to retrieve it afterward? Models Playground is here to help you do that. Models playground allows you to train your models right from the browser.  Model Zoo for AI Model Efficiency Toolkit
Model Zoo for AI Model Efficiency Toolkit

We provide a collection of popular neural network models and compare their floating point and quantized performance.

Demonstrates how to divide a DL model into multiple IR model files (division) and introduce a simplest way to implement a custom layer works with OpenVINO IR models.
Demonstrates how to divide a DL model into multiple IR model files (division) and introduce a simplest way to implement a custom layer works with OpenVINO IR models.

Demonstration of OpenVINO techniques - Model-division and a simplest-way to support custom layers Description: Model Optimizer in Intel(r) OpenVINO(tm

This project deploys a yolo fastest model in the form of tflite on raspberry 3b+. The model is from another repository of mine called -Trash-Classification-Car
This project deploys a yolo fastest model in the form of tflite on raspberry 3b+. The model is from another repository of mine called -Trash-Classification-Car

Deploy-yolo-fastest-tflite-on-raspberry 觉得有用的话可以顺手点个star嗷 这个项目将垃圾分类小车中的tflite模型移植到了树莓派3b+上面。 该项目主要是为了记录在树莓派部署yolo fastest tflite的流程 (之后有时间会尝试用C++部署来提升

MBPO (paper: When to trust your model: Model-based policy optimization) in offline RL settings

offline-MBPO This repository contains the code of a version of model-based RL algorithm MBPO, which is modified to perform in offline RL settings Pape

A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.
A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.

chitra What is chitra? chitra (चित्र) is a multi-functional library for full-stack Deep Learning. It simplifies Model Building, API development, and M

RoMA: Robust Model Adaptation for Offline Model-based Optimization

RoMA: Robust Model Adaptation for Offline Model-based Optimization Implementation of RoMA: Robust Model Adaptation for Offline Model-based Optimizatio

Comments
  • Ask about the source of active learning dataset

    Ask about the source of active learning dataset

    Hi,

    It's an interesting approach and the results are looking good in your method. But, I couldn't find any source or an explanation about your "active learning dataset". In your code, its only imports the pickled version of the data.

    Could you cite or put a link in the README for the source of active dataset?

    Thanks!

    opened by ivokun 1
  • Can't install the model with HuggingFace

    Can't install the model with HuggingFace

    Hi,

    When I am trying to import the model with HuggingFace like this :

    tokenizer = AutoTokenizer.from_pretrained("af1tang/personaGPT")
    model = AutoModelForCausalLM.from_pretrained("af1tang/personaGPT")
    
    

    I get the following error :

    Traceback (most recent call last):
      File "/home/alexis/Documents/Projets/StudioArtScience/main.py", line 7, in <module>
        tokenizer = AutoTokenizer.from_pretrained("af1tang/personaGPT")
      File "/home/alexis/anaconda3/envs/StudioArtScience/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 531, in from_pretrained
        return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
      File "/home/alexis/anaconda3/envs/StudioArtScience/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1744, in from_pretrained
        return cls._from_pretrained(
      File "/home/alexis/anaconda3/envs/StudioArtScience/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 1879, in _from_pretrained
        tokenizer = cls(*init_inputs, **init_kwargs)
      File "/home/alexis/anaconda3/envs/StudioArtScience/lib/python3.10/site-packages/transformers/models/gpt2/tokenization_gpt2_fast.py", line 137, in __init__
        super().__init__(
      File "/home/alexis/anaconda3/envs/StudioArtScience/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 108, in __init__
        fast_tokenizer = TokenizerFast.from_file(fast_tokenizer_file)
    Exception: No such file or directory (os error 2)
    

    Is there a way around this ?

    opened by AlexisPister 2
Owner
ILLIDAN Lab
Intelligent Data Analytics Lab @ MSU : Creating Large-Scale Machine Learning Algorithms for Big Data Analytics
ILLIDAN Lab
Implementation of EMNLP 2017 Paper "Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog" using PyTorch and ParlAI

Language Emergence in Multi Agent Dialog Code for the Paper Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog Satwik Kottur, José M.

Karan Desai 105 Nov 25, 2022
Code for SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations

The Second Situated Interactive MultiModal Conversations (SIMMC 2.0) Challenge 2021 Welcome to the Second Situated Interactive Multimodal Conversation

Facebook Research 81 Nov 22, 2022
VD-BERT: A Unified Vision and Dialog Transformer with BERT

VD-BERT: A Unified Vision and Dialog Transformer with BERT PyTorch Code for the following paper at EMNLP2020: Title: VD-BERT: A Unified Vision and Dia

Salesforce 44 Nov 1, 2022
Code for Talk-to-Edit (ICCV2021). Paper: Talk-to-Edit: Fine-Grained Facial Editing via Dialog.

Talk-to-Edit (ICCV2021) This repository contains the implementation of the following paper: Talk-to-Edit: Fine-Grained Facial Editing via Dialog Yumin

Yuming Jiang 221 Jan 7, 2023
In this project we investigate the performance of the SetCon model on realistic video footage. Therefore, we implemented the model in PyTorch and tested the model on two example videos.

Contrastive Learning of Object Representations Supervisor: Prof. Dr. Gemma Roig Institutions: Goethe University CVAI - Computational Vision & Artifici

Dirk Neuhäuser 6 Dec 8, 2022
Step by Step on how to create an vision recognition model using LOBE.ai, export the model and run the model in an Azure Function

Step by Step on how to create an vision recognition model using LOBE.ai, export the model and run the model in an Azure Function

El Bruno 3 Mar 30, 2022
Implementation of STAM (Space Time Attention Model), a pure and simple attention model that reaches SOTA for video classification

STAM - Pytorch Implementation of STAM (Space Time Attention Model), yet another pure and simple SOTA attention model that bests all previous models in

Phil Wang 109 Dec 28, 2022
😇A pyTorch implementation of the DeepMoji model: state-of-the-art deep learning model for analyzing sentiment, emotion, sarcasm etc

------ Update September 2018 ------ It's been a year since TorchMoji and DeepMoji were released. We're trying to understand how it's being used such t

Hugging Face 865 Dec 24, 2022
Model search is a framework that implements AutoML algorithms for model architecture search at scale

Model search (MS) is a framework that implements AutoML algorithms for model architecture search at scale. It aims to help researchers speed up their exploration process for finding the right model architecture for their classification problems (i.e., DNNs with different types of layers).

Google 3.2k Dec 31, 2022
Capture all information throughout your model's development in a reproducible way and tie results directly to the model code!

Rubicon Purpose Rubicon is a data science tool that captures and stores model training and execution information, like parameters and outcomes, in a r

Capital One 97 Jan 3, 2023