Some toy examples of score matching algorithms written in PyTorch

Ending Hsiao

Last update: Dec 26, 2022

Related tags

Deep Learning pytorch generative-model ssm toy-project dsm energy-model energy-based-model score-matching denoising-score-matching sliced-score-matching toy-score-matching

Overview

toy_gradlogp

This repo implements some toy examples of the following score matching algorithms in PyTorch:

ssm-vr: sliced score matching with variance reduction
ssm: sliced score matching
deen: deep energy estimator networks
dsm: denoisnig score matching

Installation

Basic requirements:

Python >= 3.6
TensorFlow >= 2.3.0
PyTorch >= 1.8.0

Install from PyPI

pip install toy_gradlogp

Or install the latest version from this repo

pip install git+https://github.com.Ending2015a/toy_gradlogp.git@master

Examples

The examples are placed in toy_gradlogp/run/

Train an energy model

Run ssm-vr on 2spirals dataset (don't forget to add --gpu to enable gpu)

python -m toy_gradlogp.run.train_energy --gpu --loss ssm-vr --data 2spirals

To see the full options, type --help command:

python -m toy_gradlogp.run.train_energy --help

usage: train_energy.py [-h] [--logdir LOGDIR]
                       [--data {8gaussians,2spirals,checkerboard,rings}]
                       [--loss {ssm-vr,ssm,deen,dsm}]
                       [--noise {radermacher,sphere,gaussian}] [--lr LR]
                       [--size SIZE] [--eval_size EVAL_SIZE]
                       [--batch_size BATCH_SIZE] [--n_epochs N_EPOCHS]
                       [--n_slices N_SLICES] [--n_steps N_STEPS] [--eps EPS]
                       [--gpu] [--log_freq LOG_FREQ] [--eval_freq EVAL_FREQ]
                       [--vis_freq VIS_FREQ]

optional arguments:
  -h, --help            show this help message and exit
  --logdir LOGDIR
  --data {8gaussians,2spirals,checkerboard,rings}
                        dataset
  --loss {ssm-vr,ssm,deen,dsm}
                        loss type
  --noise {radermacher,sphere,gaussian}
                        noise type
  --lr LR               learning rate
  --size SIZE           dataset size
  --eval_size EVAL_SIZE
                        dataset size for evaluation
  --batch_size BATCH_SIZE
                        training batch size
  --n_epochs N_EPOCHS   number of epochs to train
  --n_slices N_SLICES   number of slices for sliced score matching
  --n_steps N_STEPS     number of steps for langevin dynamics
  --eps EPS             noise scale for langevin dynamics
  --gpu                 enable gpu
  --log_freq LOG_FREQ   logging frequency (unit: epoch)
  --eval_freq EVAL_FREQ
                        evaluation frequency (unit: epoch)
  --vis_freq VIS_FREQ   visualization frequency (unit: epoch)

Results

Tips: The larger density has a lower energy!

`8gaussians`

Algorithm	Results
`ssm-vr`
`ssm`
`deen`
`dsm`

`2spirals`

Algorithm	Results
`ssm-vr`
`ssm`
`deen`
`dsm`

`checkerboard`

Algorithm	Results
`ssm-vr`
`ssm`
`deen`
`dsm`

`rings`

Algorithm	Results
`ssm-vr`
`ssm`
`deen`
`dsm`

Many Class Activation Map methods implemented in Pytorch for CNNs and Vision Transformers. Including Grad-CAM, Grad-CAM++, Score-CAM, Ablation-CAM and XGrad-CAM

Class Activation Map methods implemented in Pytorch pip install grad-cam ⭐ Tested on many Common CNN Networks and Vision Transformers. ⭐ Includes smoo

6.6k Jan 6, 2023

PyTorch implementation for Score-Based Generative Modeling through Stochastic Differential Equations (ICLR 2021, Oral)

Score-Based Generative Modeling through Stochastic Differential Equations This repo contains a PyTorch implementation for the paper Score-Based Genera

757 Jan 4, 2023

Scripts of Machine Learning Algorithms from Scratch. Implementations of machine learning models and algorithms using nothing but NumPy with a focus on accessibility. Aims to cover everything from basic to advance.

Algo-ScriptML Python implementations of some of the fundamental Machine Learning models and algorithms from scratch. The goal of this project is not t

81 Nov 26, 2022

Official code for Score-Based Generative Modeling through Stochastic Differential Equations

Score-Based Generative Modeling through Stochastic Differential Equations This repo contains the official implementation for the paper Score-Based Gen

818 Jan 6, 2023

Official implementation for Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder at NeurIPS 2020

Likelihood-Regret Official implementation of Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder at NeurIPS 2020. T

33 Oct 12, 2022

Repository for the "Gotta Go Fast When Generating Data with Score-Based Models" paper

Gotta Go Fast When Generating Data with Score-Based Models This repo contains the official implementation for the paper Gotta Go Fast When Generating

89 Nov 9, 2022

Code for paper: Group-CAM: Group Score-Weighted Visual Explanations for Deep Convolutional Networks

Group-CAM By Zhang, Qinglong and Rao, Lu and Yang, Yubin [State Key Laboratory for Novel Software Technology at Nanjing University] This repo is the o

98 Nov 16, 2022

The first dataset of composite images with rationality score indicating whether the object placement in a composite image is reasonable.

Object-Placement-Assessment-Dataset-OPA Object-Placement-Assessment (OPA) is to verify whether a composite image is plausible in terms of the object p

53 Nov 15, 2022

Definition of a business problem according to Wilson Lower Bound Score and Time Based Average Rating

Wilson Lower Bound Score, Time Based Rating Average In this study I tried to calculate the product rating and sorting reviews more accurately. I have

3 Sep 30, 2021

Some toy examples of score matching algorithms written in PyTorch

Related tags

Overview

toy_gradlogp

Installation

Examples

Train an energy model

Results

`8gaussians`

`2spirals`

`checkerboard`

`rings`

You might also like...

Many Class Activation Map methods implemented in Pytorch for CNNs and Vision Transformers. Including Grad-CAM, Grad-CAM++, Score-CAM, Ablation-CAM and XGrad-CAM

PyTorch implementation for Score-Based Generative Modeling through Stochastic Differential Equations (ICLR 2021, Oral)

Scripts of Machine Learning Algorithms from Scratch. Implementations of machine learning models and algorithms using nothing but NumPy with a focus on accessibility. Aims to cover everything from basic to advance.

Official code for Score-Based Generative Modeling through Stochastic Differential Equations

Official implementation for Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder at NeurIPS 2020

Repository for the "Gotta Go Fast When Generating Data with Score-Based Models" paper

Code for paper: Group-CAM: Group Score-Weighted Visual Explanations for Deep Convolutional Networks

The first dataset of composite images with rationality score indicating whether the object placement in a composite image is reasonable.

Definition of a business problem according to Wilson Lower Bound Score and Time Based Average Rating

Owner

Ending Hsiao

ColossalAI-Examples - Examples of training models with hybrid parallelism using ColossalAI

A toy project using OpenCV and PyMunk

This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of Coordinate Independent Convolutional Networks.

A toy compiler that can convert Python scripts to pickle bytecode 🥒

GAN JAX - A toy project to generate images from GANs with JAX

Pytoydl: A toy deep learning framework built upon numpy.

Code for C2-Matching (CVPR2021). Paper: Robust Reference-based Super-Resolution via C2-Matching.

A Python implementation of the Locality Preserving Matching (LPM) method for pruning outliers in image matching.

Independent and minimal implementations of some reinforcement learning algorithms using PyTorch (including PPO, A3C, A2C, ...).

Pytorch Implementations of large number classical backbone CNNs, data enhancement, torch loss, attention, visualization and some common algorithms.