Introduction to CPM

Tsinghua AI

Last update: Dec 23, 2022

Related tags

Deep Learning CPM

Overview

CPM

CPM is an open-source program on large-scale pre-trained models, which is conducted by Beijing Academy of Artificial Intelligence and Tsinghua University, with the goal of building large-scale Chinese-centered pre-trained models. The open-source models can be widely used in Chinese natural language understanding, generative tasks, and all of them are free and open for download for research use.

CPM-1

CPM: A Large-scale Generative Chinese Pre-trained Language Model. [paper]

Codes:

Models: [Download]

CPM-2

CPM-2: Large-scale Cost-effective Pre-trained Language Models. [paper]

Codes:

Models: [Download]

PLM Survey

Pre-Trained Models: Past, Present and Future. [paper]

Useful Links

Article Generation with CPM-1. [link]

Cite

@article{cpm-v1,
  title={CPM: A Large-scale Generative Chinese Pre-trained Language Model},
  author={Zhang, Zhengyan and Han, Xu, and Zhou, Hao, and Ke, Pei, and Gu, Yuxian and Ye, Deming and Qin, Yujia and Su, Yusheng and Ji, Haozhe and Guan, Jian and Qi, Fanchao and Wang, Xiaozhi and Zheng, Yanan and Zeng, Guoyang and Cao, Huanqi and Chen, Shengqi and Li, Daixuan and Sun, Zhenbo and Liu, Zhiyuan and Huang, Minlie and Han, Wentao and Tang, Jie and Li, Juanzi and Sun, Maosong},
  year={2020}
}

@article{cpm-v2,
  title={CPM-2: Large-scale Cost-efficient Pre-trained Language Models},
  author={Zhang, Zhengyan and Gu, Yuxian and Han, Xu and Chen, Shengqi and Xiao, Chaojun and Sun, Zhenbo and Yao, Yuan and Qi, Fanchao and Guan, Jian and Ke, Pei and Cai, Yanzheng and Zeng, Guoyang and Tan, Zhixing and Liu, Zhiyuan and Huang, Minlie and Han, Wentao and Liu, Yang and Zhu, Xiaoyan and Sun, Maosong},
  year={2021}
}

@article{han2021pretrained,
      title={Pre-Trained Models: Past, Present and Future}, 
      author={Xu Han and Zhengyan Zhang and Ning Ding and Yuxian Gu and Xiao Liu and Yuqi Huo and Jiezhong Qiu and Liang Zhang and Wentao Han and Minlie Huang and Qin Jin and Yanyan Lan and Yang Liu and Zhiyuan Liu and Zhiwu Lu and Xipeng Qiu and Ruihua Song and Jie Tang and Ji-Rong Wen and Jinhui Yuan and Wayne Xin Zhao and Jun Zhu},
      year={2021}
}

Easy-to-use CPM for Chinese text generation

CPM 项目描述 CPM（Chinese Pretrained Models）模型是北京智源人工智能研究院和清华大学发布的中文大规模预训练模型。官方发布了三种规模的模型，参数量分别为109M、334M、2.6B，用户需申请与通过审核，方可下载。由于原项目需要考虑大模型的训练和使用，需要安装较为复杂

382 Jan 7, 2023

Code for CPM-2 Pre-Train

CPM-2 Pre-Train Pre-train CPM-2 此分支为110亿非 MoE 模型的预训练代码，MoE 模型的预训练代码请切换到 moe 分支 CPM-2技术报告请参考link。 0 模型下载请在智源资源下载页面进行申请，文件介绍如下：文件名描述参数大小 100000.tar

136 Dec 28, 2022

Chinese Pre-Trained Language Models (CPM-LM) Version-I

CPM-Generate 为了促进中文自然语言处理研究的发展，本项目提供了 CPM-LM (2.6B) 模型的文本生成代码，可用于文本生成的本地测试，并以此为基础进一步研究零次学习/少次学习等场景。[项目首页] [模型下载] [技术报告] 若您想使用CPM-1进行推理，我们建议使用高效推理工具BMI

1.4k Jan 3, 2023

1st-in-MICCAI2020-CPM - Combined Radiology and Pathology Classification

Combined Radiology and Pathology Classification MICCAI 2020 Combined Radiology a

22 Dec 8, 2022

aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)

Bayesian Methods for Hackers Using Python and PyMC The Bayesian method is the natural approach to inference, yet it is hidden from readers behind chap

25.1k Jan 2, 2023

An introduction of Markov decision process (MDP) and two algorithms that solve MDPs (value iteration, policy iteration) along with their Python implementations.

Markov Decision Process A Markov decision process (MDP), by definition, is a sequential decision problem for a fully observable, stochastic environmen

31 Dec 30, 2022

A Pythonic introduction to methods for scaling your data science and machine learning work to larger datasets and larger models, using the tools and APIs you know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

This tutorial's purpose is to introduce Pythonistas to methods for scaling their data science and machine learning work to larger datasets and larger models, using the tools and APIs they know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

102 Nov 10, 2022

Introduction to WebScraping Workshop - Semcomp 24 Beta

Extrair informações da internet de forma automatizada. Existem diversas maneiras de fazer isso, nesse tutorial vamos ver algumas delas, por meio de bibliotecas de python.

19 Sep 11, 2022

Introduction to Django Rest Framework

Introduction to Django Rest Framework This is the repository of the video series Introduction to Django Rest Framework published on YouTube. It is a s

20 Jul 14, 2022

Introduction to Augmented Reality (AR) with Python 3 and OpenCV 4.2.

85 Jan 2, 2023

A repository containing an introduction to Panel made to be support videos and talks.

👍 Awesome Panel - Introduction to Panel THIS REPO IS WORK IN PROGRESS. PRE-ALPHA Panel is a very powerful framework for exploratory data analysis and

51 Nov 17, 2022

A PyTorch implementation of Radio Transformer Networks from the paper "An Introduction to Deep Learning for the Physical Layer".

An Introduction to Deep Learning for the Physical Layer An usable PyTorch implementation of the noisy autoencoder infrastructure in the paper "An Intr

120 Nov 21, 2022

An introduction to hikari, complete with different examples for different command handlers.

An intro to hikari This repo provides some simple examples to get you started with hikari. Contained in this repo are bots designed with both the hika

18 Nov 29, 2022

Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark

SiamSA: Robust Siamese Object Tracking for Unmanned Aerial Manipulator Demo video 📹 Our video on Youtube and bilibili demonstrates the evaluation of

Intelligent Vision for Robotics in Complex Environment

12 Dec 18, 2022

Comments

Missing References in the Prompt Tuning section.
There is a large body of prior and concurrent work in prompt tuning that has been left out of this paper. For example:

Learning How to Ask, one of the first to learn a continuous version of a prompt https://arxiv.org/abs/2104.06599

Prefix-Tuning https://arxiv.org/abs/2101.00190

Prompt Tuning, where you seem to have gotten the term model-tuning from https://arxiv.org/abs/2104.08691

GPT Understands Too, which jointly learns a prompt and updates the model like the second step of your two step pipeline https://arxiv.org/abs/2103.10385

WARP https://arxiv.org/abs/2101.00121
opened by MLWatcher 1
CPM 2 198b model API

Hi, I am a student studying computer science and I'm into NLP text generation. I want to know if someone could make an API to use CPM 2 on python or the web. The model is extremely large and hosting the model somewhere would really help to avoid downloading the entire model onto a pc like mine that doesn't support Cuda or GPU acceleration. I could use a different model but I would like to use CPM 2 Moe 198b since is the latest and biggest model so far

opened by TheProtaganist 0

Introduction to CPM

Related tags

Overview

CPM

CPM-1

CPM-2

PLM Survey

Useful Links

Cite

You might also like...

Easy-to-use CPM for Chinese text generation

Code for CPM-2 Pre-Train

Chinese Pre-Trained Language Models (CPM-LM) Version-I

1st-in-MICCAI2020-CPM - Combined Radiology and Pathology Classification

aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)

An introduction of Markov decision process (MDP) and two algorithms that solve MDPs (value iteration, policy iteration) along with their Python implementations.

A Pythonic introduction to methods for scaling your data science and machine learning work to larger datasets and larger models, using the tools and APIs you know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

Introduction to WebScraping Workshop - Semcomp 24 Beta

Introduction to Django Rest Framework

Introduction to Augmented Reality (AR) with Python 3 and OpenCV 4.2.

A repository containing an introduction to Panel made to be support videos and talks.

A PyTorch implementation of Radio Transformer Networks from the paper "An Introduction to Deep Learning for the Physical Layer".

An introduction to hikari, complete with different examples for different command handlers.

Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark

Introduction to AI assignment 1 HCM University of Technology, term 211

SDU experiment of introduction to the cryptography

Introduction to Databases Coursework 2 (SQL) - dataset generator

Introduction to image processing, most used and popular functions of OpenCV

An introduction course for Python provided by VetsInTech

Comments

Missing References in the Prompt Tuning section.

CPM 2 198b model API

Owner

Tsinghua AI

Code for CPM-2 Pre-Train

1st-in-MICCAI2020-CPM - Combined Radiology and Pathology Classification

aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)

A PyTorch implementation of Radio Transformer Networks from the paper "An Introduction to Deep Learning for the Physical Layer".

Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark

Introduction to AI assignment 1 HCM University of Technology, term 211

CS50's Introduction to Artificial Intelligence Test Scripts

[AI6101] Introduction to AI & AI Ethics is a core course of MSAI, SCSE, NTU, Singapore

Pretrain CPM - 大规模预训练语言模型的预训练代码

Code for CPM-2 Pre-Train