Perturb-and-max-product: Sampling and learning in discrete energy-based models

Overview

Perturb-and-max-product: Sampling and learning in discrete energy-based models

This repo contains code for reproducing the results in the paper Perturb-and-max-product: Sampling and learning in discrete energy-based models accepted at the 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

Getting started

Dependencies can be installed via

pip install -r requirements.txt
python setup.py develop

By default this installs JAX for CPU. If you would like to use JAX with a GPU and specific CUDA version (highly recommended), follow the official instructions here.

Pmap

pmap is the main folder. It contains the following files:

  • mmd.py implements the maximum mean discrepancy metric.
  • small_ising_scoring.py contains useful functions for small tractable Ising models.
  • ising_modeling.py contains learning and sampling algorithms for Ising models using max-product and gibbs variants (in JAX).
  • ising_modeling_lp.py contains similar algorithms using Ecos LP solver.
  • mplp.py implements the max-product linear programming algorithm for Ising models.
  • rbm_modeling.py contains learning and sampling algorithms for RBM models using max-product and gibbs variants (in JAX).
  • rbm_modeling_lp.py contains similar algorithms using Ecos LP solver.
  • conv_or_modeling.py and logical_mpmp.py contain sampling algorithms for the deconvolution experiments in Section 5.6.

Experiments

The experiments folder contains the python scripts used for all the experiments the paper.

The data required for all the experiments has to be generated first via

. experiments/generate_data.sh

and will be automatically stored in a data folder

  • Experiments for Section 5.1 are in exp1_wrongmodel.py.
  • Experiments for Section 5.2 are in exp2_mplp.py.
  • Experiments for Section 5.3 are in exp3_zeros_train.py and exp3_zeros_test.py.
  • Experiments for Section 5.4 are in exp4_c2d_lattice_persistent.py, exp4_c2d_lattice_non_persistent.py, exp_erdos_persistent.py andexp_erdos_non_persistent.py.
  • Experiments for Section 5.5 are in exp5_mnist_train.py, exp5_mnist_test.py and exp5_rbm_2s.py.
  • Experiments for Section 5.6 are in exp6_convor.py.

The results will be automatically stored in a results folder

Figures

The notebook all_paper_plots.ipynb displays all the figures of the main paper. The figures are saved in a paper folder.

You might also like...
Official codes for the paper
Official codes for the paper "Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech"

ResDAVEnet-VQ Official PyTorch implementation of Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech What is in this repo? M

Drslmarkov - Distributionally Robust Structure Learning for Discrete Pairwise Markov Networks

Distributionally Robust Structure Learning for Discrete Pairwise Markov Networks

Code for ICLR 2021 Paper,
Code for ICLR 2021 Paper, "Anytime Sampling for Autoregressive Models via Ordered Autoencoding"

Anytime Autoregressive Model Anytime Sampling for Autoregressive Models via Ordered Autoencoding , ICLR 21 Yilun Xu, Yang Song, Sahaj Gara, Linyuan Go

Official PyTorch implementation for FastDPM, a fast sampling algorithm for diffusion probabilistic models

Official PyTorch implementation for "On Fast Sampling of Diffusion Probabilistic Models". FastDPM generation on CIFAR-10, CelebA, and LSUN datasets. S

the code for paper "Energy-Based Open-World Uncertainty Modeling for Confidence Calibration"

EOW-Softmax This code is for the paper "Energy-Based Open-World Uncertainty Modeling for Confidence Calibration". Accepted by ICCV21. Usage Commnd exa

Energy consumption estimation utilities for Jetson-based platforms

This repository contains a utility for measuring energy consumption when running various programs in NVIDIA Jetson-based platforms. Currently TX-2, NX, and AGX are supported.

Auto HMM: Automatic Discrete and Continous HMM including Model selection

Auto HMM: Automatic Discrete and Continous HMM including Model selection

PyTorch package for the discrete VAE used for DALL·E.

Overview [Blog] [Paper] [Model Card] [Usage] This is the official PyTorch package for the discrete VAE used for DALL·E. Installation Before running th

DCT-Mask: Discrete Cosine Transform Mask Representation for Instance Segmentation

DCT-Mask: Discrete Cosine Transform Mask Representation for Instance Segmentation This project hosts the code for implementing the DCT-MASK algorithms

Owner
Vicarious
Vicarious
Invert and perturb GAN images for test-time ensembling

GAN Ensembling Project Page | Paper | Bibtex Ensembling with Deep Generative Views. Lucy Chai, Jun-Yan Zhu, Eli Shechtman, Phillip Isola, Richard Zhan

Lucy Chai 93 Dec 8, 2022
Invert and perturb GAN images for test-time ensembling

Invert and perturb GAN images for test-time ensembling

Lucy Chai 49 May 2, 2021
Streamlit App For Product Analysis - Streamlit App For Product Analysis

Streamlit_App_For_Product_Analysis Здравствуйте! Перед вами дашборд, позволяющий

Grigory Sirotkin 1 Jan 10, 2022
PyTorch implementation of Algorithm 1 of "On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models"

Code for On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models This repository will reproduce the main results from our pape

Mitch Hill 32 Nov 25, 2022
💡 Learnergy is a Python library for energy-based machine learning models.

Learnergy: Energy-based Machine Learners Welcome to Learnergy. Did you ever reach a bottleneck in your computational experiments? Are you tired of imp

Gustavo Rosa 57 Nov 17, 2022
Learning Energy-Based Models by Diffusion Recovery Likelihood

Learning Energy-Based Models by Diffusion Recovery Likelihood Ruiqi Gao, Yang Song, Ben Poole, Ying Nian Wu, Diederik P. Kingma Paper: https://arxiv.o

Ruiqi Gao 41 Nov 22, 2022
PyTorch code accompanying our paper on Maximum Entropy Generators for Energy-Based Models

Maximum Entropy Generators for Energy-Based Models All experiments have tensorboard visualizations for samples / density / train curves etc. To run th

Rithesh Kumar 135 Oct 27, 2022
Code for "Adversarial Attack Generation Empowered by Min-Max Optimization", NeurIPS 2021

Min-Max Adversarial Attacks [Paper] [arXiv] [Video] [Slide] Adversarial Attack Generation Empowered by Min-Max Optimization Jingkang Wang, Tianyun Zha

Jingkang Wang 12 Nov 23, 2022