Multiwavelets-based operator model

Last update: Dec 4, 2022

Overview

Multiwavelet model for Operator maps

Gaurav Gupta, Xiongye Xiao, and Paul Bogdan
Multiwavelet-based Operator Learning for Differential Equations
In NeurIPS 2021. arXiv:2109.13459

Setup

Requirements

The code package is developed using Python 3.8 and Pytorch 1.8 with cuda 11.0. For running the experiments first install the required packages using 'requirements.txt'

Experiments

Data

Generate the data using the scripts provided in the 'Data' directory. The scripts use Matlab 2018+. A sample generated dataset for KdV is uploaded at KdV data.

For the experiments on Burgers, Darcy, and Navier Stokes, the code package uses the datasets as provided in the following repository by the Authors Zongyi Li et al.

PDE datasets

Scripts

Choose the required model from the models (1-d, 2-d, 2-d time-varying) and pass-in the required polynomial: 'legendre' or 'chebyshev'. Next, choose the desired value of multiwavelets 'k'.

kDV

As an example, a complete pipeline is shown for the kDV equation in the attached kDV.ipynb notebook.

Navier Stokes

The pre-trained models for Navier Stokes equation is provided using the following link:

NS Pre trained

A visual of time-evolution of the estimated outputs of the pre-trained models is available Here.

To test the model, first download the models to the 'ptmodels' directory. Next, For N=1000, T = 50, \nu = 1e-3

python test_NS_MWT_N_1000.py

For N = 10000, T = 30, \nu = 1e-4

python test_NS_MWT_N_10000.py

Note: The NS experiments were done using Pytorch 1.7 cuda 11.0

Citation

If you use this code, or our work, please cite:

@misc{gupta2021multiwavelet,
      title={Multiwavelet-based Operator Learning for Differential Equations}, 
      author={Gaurav Gupta and Xiongye Xiao and Paul Bogdan},
      year={2021},
      eprint={2109.13459},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

Comments

What version of Fourier Neural Operator is used in the MWT paper?

Hi, thanks for the sharing the nicely written code on the wavelet neural operator.

I wonder what version of FNO is used in the comparison? for example in Table 2 of https://arxiv.org/abs/2109.13459v1

Because in Table 2, FNO1d still has only 1e-2 evaluation relative error for the Burgers' benchmark.

In the new implementation, if BatchNorm is removed, and change the ReLU activation to GeLU or SiLU, 500 epochs of MultiStepLR scheduling will yield about 2.5e-3 to 2.7e-3 evaluation relative error: see zongyi-li/fourier_neural_operator#30 , even with a 1cycleLR + 100 epochs, the evaluation error is around 4.1e-3.

opened by scaomath 2
Dealing with data with resolutions which are not a power of 2?
https://github.com/gaurav71531/mwt-operator/blob/4b0770c0987a325b5a18775a0bdb63ab6d69435e/models/models.py#L141

This can be seen at the start of each wavelet_transform function.

For example, if x has shape (B, 65, 65, W), the concatenation returns an error, because on the second axis, the slice ::2 and 1::2 return different dimensions.

xa = torch.cat([x[:, ::2, :, :], x[:, 1::2, :, :], ], -1)

Thanks for sharing these very nice results.
opened by dericknganyu 1

ReConsider is a re-ranking model that re-ranks the top-K (passage, answer-span) predictions of an Open-Domain QA Model like DPR (Karpukhin et al., 2020).

ReConsider ReConsider is a re-ranking model that re-ranks the top-K (passage, answer-span) predictions of an Open-Domain QA Model like DPR (Karpukhin

47 Jul 26, 2022

Ever felt tired after preprocessing the dataset, and not wanting to write any code further to train your model? Ever encountered a situation where you wanted to record the hyperparameters of the trained model and able to retrieve it afterward? Models Playground is here to help you do that. Models playground allows you to train your models right from the browser.

Models Playground 🗂️ Upload a Preprocessed Dataset 🌠 Choose whether to perform Classification or Regression 🦹 Enter the Dependent Variable ?

19 Dec 10, 2022

Model Zoo for AI Model Efficiency Toolkit

We provide a collection of popular neural network models and compare their floating point and quantized performance.

137 Jan 3, 2023

😇A pyTorch implementation of the DeepMoji model: state-of-the-art deep learning model for analyzing sentiment, emotion, sarcasm etc

------ Update September 2018 ------ It's been a year since TorchMoji and DeepMoji were released. We're trying to understand how it's being used such t

865 Dec 24, 2022

This repo uses a combination of logits and feature distillation method to teach the PSPNet model of ResNet18 backbone with the PSPNet model of ResNet50 backbone. All the models are trained and tested on the PASCAL-VOC2012 dataset.

PSPNet-logits and feature-distillation Introduction This repository is based on PSPNet and modified from semseg and Pixelwise_Knowledge_Distillation_P

6 Dec 1, 2022

Demonstrates how to divide a DL model into multiple IR model files (division) and introduce a simplest way to implement a custom layer works with OpenVINO IR models.

Demonstration of OpenVINO techniques - Model-division and a simplest-way to support custom layers Description: Model Optimizer in Intel(r) OpenVINO(tm

12 Nov 9, 2022

This project deploys a yolo fastest model in the form of tflite on raspberry 3b+. The model is from another repository of mine called -Trash-Classification-Car

Deploy-yolo-fastest-tflite-on-raspberry 觉得有用的话可以顺手点个star嗷这个项目将垃圾分类小车中的tflite模型移植到了树莓派3b+上面。该项目主要是为了记录在树莓派部署yolo fastest tflite的流程 (之后有时间会尝试用C++部署来提升

7 Aug 16, 2022

A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.

chitra What is chitra? chitra (चित्र) is a multi-functional library for full-stack Deep Learning. It simplifies Model Building, API development, and M

210 Dec 21, 2022

Arch-Net: Model Distillation for Architecture Agnostic Model Deployment

Arch-Net: Model Distillation for Architecture Agnostic Model Deployment The official implementation of Arch-Net: Model Distillation for Architecture A

22 Jan 5, 2023

Multiwavelets-based operator model

Related tags

Overview

Multiwavelet model for Operator maps

Setup

Requirements

Experiments

Data

Scripts

kDV

Navier Stokes

Citation

You might also like...

ReConsider is a re-ranking model that re-ranks the top-K (passage, answer-span) predictions of an Open-Domain QA Model like DPR (Karpukhin et al., 2020).

Model Zoo for AI Model Efficiency Toolkit

😇A pyTorch implementation of the DeepMoji model: state-of-the-art deep learning model for analyzing sentiment, emotion, sarcasm etc

This repo uses a combination of logits and feature distillation method to teach the PSPNet model of ResNet18 backbone with the PSPNet model of ResNet50 backbone. All the models are trained and tested on the PASCAL-VOC2012 dataset.

Demonstrates how to divide a DL model into multiple IR model files (division) and introduce a simplest way to implement a custom layer works with OpenVINO IR models.

This project deploys a yolo fastest model in the form of tflite on raspberry 3b+. The model is from another repository of mine called -Trash-Classification-Car

A multi-functional library for full-stack Deep Learning. Simplifies Model Building, API development, and Model Deployment.

Arch-Net: Model Distillation for Architecture Agnostic Model Deployment

Comments

What version of Fourier Neural Operator is used in the MWT paper?

Dealing with data with resolutions which are not a power of 2?

Owner

Gaurav

DI-HPC is an acceleration operator component for general algorithm modules in reinforcement learning algorithms

In this project we investigate the performance of the SetCon model on realistic video footage. Therefore, we implemented the model in PyTorch and tested the model on two example videos.

Step by Step on how to create an vision recognition model using LOBE.ai, export the model and run the model in an Azure Function

MBPO (paper: When to trust your model: Model-based policy optimization) in offline RL settings

RoMA: Robust Model Adaptation for Offline Model-based Optimization

An atmospheric growth and evolution model based on the EVo degassing model and FastChem 2.0

A lightweight Python-based 3D network multi-agent simulator. Uses a cell-based congestion model. Calculates risk, loudness and battery capacities of the agents. Suitable for 3D network optimization tasks.

Model search is a framework that implements AutoML algorithms for model architecture search at scale

Capture all information throughout your model's development in a reproducible way and tie results directly to the model code!

Implementation of STAM (Space Time Attention Model), a pure and simple attention model that reaches SOTA for video classification