Physics-informed Neural Operator for Learning Partial Differential Equation

Last update: Jan 2, 2023

Related tags

Deep Learning PINO

Overview

PINO

Physics-informed Neural Operator for Learning Partial Differential Equation

Abstract: Machine learning methods have recently shown promise in solving partial differential equations (PDEs). They can be classified into two broad categories: solution function approximation and operator learning. The Physics-Informed Neural Network (PINN) is an example of the former while the Fourier neural operator (FNO) is an example of the latter. Both these approaches have shortcomings. The optimization in PINN is challenging and prone to failure, especially on multi-scale dynamic systems. FNO does not suffer from this optimization issue since it carries out supervised learning on a given dataset, but obtaining such data may be too expensive or infeasible. In this work, we propose the physics-informed neural operator (PINO), where we combine the operating-learning and function-optimization frameworks, and this improves convergence rates and accuracy over both PINN and FNO models. In the operator-learning phase, PINO learns the solution operator over multiple instances of the parametric PDE family. In the test-time optimization phase, PINO optimizes the pre-trained operator ansatz for the querying instance of the PDE. Experiments show PINO outperforms previous ML methods on many popular PDE families while retaining the extraordinary speed-up of FNO compared to solvers. In particular, PINO accurately solves long temporal transient flows and Kolmogorov flows, while PINN and other methods fail to converge.

Requirements

Pytorch 1.8.0 or later
wandb
tqdm
scipy
h5py
numpy
DeepXDE:latest
tensorflow 2.4.0

Data description

Burgers equation

burgers_pino.mat

Darcy flow

spatial domain: $x\in (0,1)^2$
Data file: piececonst_r421_N1024_smooth1.mat, piececonst_r421_N1024_smooth2.mat
Raw data shape: 1024x421x421

Long roll out of Navier Stokes equation

spatial domain: $x\in (0, 1)^2$
temporal domain: $t\in [0, 49]$
forcing: $0.1(\sin(2\pi(x_1+x_2)) + \cos(2\pi(x_1+x_2)))$
viscosity = 0.001

Data file: nv_V1e-3_N5000_T50.mat, with shape 50 x 64 x 64 x 5000

train set: -1-4799
test set: 4799-4999

Navier Stokes with Reynolds number 500

spatial domain: $x\in (0, 2\pi)^2$
temporal domain: $t \in [0, 0.5]$
forcing: $-4\cos(4x_2)$
Reynolds number: 500

Train set: data of shape (N, T, X, Y) where N is the number of instances, T is temporal resolution, X, Y are spatial resolutions.

NS_fft_Re500_T4000.npy : 4000x64x64x65
NS_fine_Re500_T128_part0.npy: 100x129x128x128
NS_fine_Re500_T128_part1.npy: 100x129x128x128

Test set: data of shape (N, T, X, Y) where N is the number of instances, T is temporal resolution, X, Y are spatial resolutions.

NS_Re500_s256_T100_test.npy: 100x129x256x256
NS_fine_Re500_T128_part2.npy: 100x129x128x128

Configuration file format: see .yaml files under folder configs for detail.

Code for Burgers equation

Train PINO

To run PINO for Burgers equation, use, e.g.,

python3 train_burgers.py --config_path configs/pretrain/burgers-pretrain.yaml --mode train

To test PINO for burgers equation, use, e.g.,

python3 train_burgers.py --config_path configs/test/burgers.yaml --mode test

Code for Darcy Flow

Operator learning

To run PINO for Darcy Flow, use, e.g.,

python3 train_operator.py --config_path configs/pretrain/Darcy-pretrain.yaml

To evaluate operator for Darcy Flow, use, e.g.,

python3 eval_operator.py --config_path configs/test/darcy.yaml

Test-time optimization

To do test-time optimization for Darcy Flow, use, e.g.,

python3 run_pino2d.py --config_path configs/finetune/Darcy-finetune.yaml --start [starting index] --stop [stopping index]

Baseline

To run DeepONet, use, e.g.,

python3 deeponet.py --config_path configs/pretrain/Darcy-pretrain-deeponet.yaml --mode train

To test DeepONet, use, e.g.,

python3 deeponet.py --config_path configs/test/darcy.yaml --mode test

Code for Navier Stokes equation

Train PINO for short time period

To run operator learning, use, e.g.,

python3 train_operator.py --config_path configs/pretrain/Re500-pretrain-05s-4C0.yaml

To evaluate trained operator, use

python3 eval_operator.py --config_path configs/test/Re500-05s.yaml

To run test-time optimization, use

python3 train_PINO3d.py --config_path configs/***.yaml

To train Navier Stokes equations sequentially without running train_PINO3d.py multiple times, use

python3 run_pino3d.py --config_path configs/[configuration file name].yaml --start [index of the first data] --stop [which data to stop]

Baseline for short time period

To train DeepONet, use

python3 deeponet.py --config_path configs/[configuration file].yaml --mode train

To test DeepONet, use

python3 deeponet.py --config_path configs/[configuration file].yaml --mode test

To train and test PINNs, use, e.g.,

python3 nsfnet.py --config_path configs/Re500-pinns-05s.yaml --start [starting index] --stop [stopping index]

Baseline for long roll out

To train and test PINNs, use

python3 nsfnet.py --config_path configs/scratch/NS-50s.yaml --long --start [starting index] --stop [stopping index]

Pseudospectral solver for Navier Stokes equation

To run solver, use

python3 run_solver.py --config_path configs/Re500-0.5s.yaml

Comments

A question about PDE loss for PINO (Burgers equation)

Dear devzhk, I'm sorry to bother you. I have been reading your code for PINO on Github and I have a question about the code for computing the PDE loss on fourier domain (for Burgers equation).The original code (FDM_Burgers in losses.py) suggest that the derivative of the signal can be computed with: ux = irfft{2jpik_xfft(u)} I wasn't very familiar with the derivative feature of the discrete fourier transformations. So I tested it with a simple cos function to verify the code (Ground true should be sin function). However, the test seem to reflect that the derivative of the signal should be computed with: ux = irfft{1jk_x*fft(u)} I'm not sure if I was wrong, I have uploaded the code in the txt file, can you elaborate it? Again, I am sorry to bother you TestCode.txt .

opened by whDingGit 3
Files can't download

Hi Devzhk,

When I download the piececonst_r421_N1024_smooth1.mat file, the web page shows This XML file does not appear to have any style information associated with it. The document tree is shown below.

Is there any other way to download the dataset?

Thanks, Nevo liu

opened by nevoliu 1
Some of the problems about NS_fine_Re500_T128_part0.npy

interesting work！

I'm confused about the shape of x in NS_fine_Re500_T128_part0.npy x torch.Size([1, 128, 128, 129, 4]) y torch.Size([1, 128, 128, 129]) why the shape of x is [1, 128, 128, 129, 4] instead of [1, 128, 128, 129] What does the number 4 mean？

opened by qxj233333 1
MATLAB script to generate the Darcy dataset

Hi, thanks again for sharing the code and links to the datasets. Could you please also share the MATLAB script used to generate the Darcy flow dataset ?

Thanks! :)

opened by AhmedFrikha 1
Test set used for Darcy experiments: "piececonst_r421_N1024_smooth2.mat"

Hi, thanks for sharing the code and the link to the training sets. Could you please also share the link to the test set used for the Darcy experiments "piececonst_r421_N1024_smooth2.mat" ?

Thanks !!

opened by AhmedFrikha 1
piececonst_r421_N1024_smooth1.mat?

Hello, I was trying to train PINO on Burger and on Darcy but I obtain the following error.

FileNotFoundError: [Errno 2] No such file or directory: '/mnt/md1/zongyi/piececonst_r421_N1024_smooth1.mat'

I'm not able to understand where that is in the code and or where to find the file. Any suggestions?

Best

Marco

opened by mgraffiedi 1
Helmholtz PDE

Dear Hongkai,

Thank you for your very interesting research. Last half a year, I was aiming to solve the Helmholtz equation with PINNs using DeepXDE packages. I could reach a mean average percentage error of 0.06% for some cases. Now I am looking for a proper approach to solve the parametric Helmholtz PDE with up to 20 extra input parameters besides the spatial coordinates to develop a generalized PINN. From experiments, PINN itself handled parametric PDE with worse accuracy, so this makes me have a look at PINO. Q1: Transferring PINN to PINO is quite painful for me. Do you have a successful experience with training PINN on multidimensional input? At the moment, it looks like all my attempts are in vain, so do you know if PINO/FNO/DeepONet is the better way to solve parametric PDEs? Q2: My boundary conditions are hard-constrained periodic ones, so no exact values are known like in the case of Burger's equation. I cannot do purely-physics-informed training in such a case and have to feed the operator with target data?

opened by alabaykazakh 1
Code for Darcy Flow does not work

Hi,

Amazing project! I am able to duplicate the Burgers experiment. However, the code for Darcy Flow does not work.

python3 train_operator.py --config_path configs/pretrain/Darcy-pretrain.yaml

with the following error: Traceback (most recent call last): File "train_operator.py", line 133, in train_2d(args, config) File "train_operator.py", line 90, in train_2d num=data_config['n_samples'], KeyError: 'n_samples'

opened by Shancong-Mou 3
Cavity Flow Test Cases
Amazing project. I have been able to reproduce some of the Kolmogorov flows. However, I am more interested in the cavity flow scenarios. Would you be able to supply:

Training and Testing data

Yaml configs

Any additional functions or changes to the losses (as I am guessing you would have had to change those).

I really want to try and replicate your cavity flow results with PINO.
opened by cfos3120 0

Owner

GitHub https://devzhk.github.io/PINO/

Must-read Papers on Physics-Informed Neural Networks.

PINNpapers Contributed by IDRL lab. Introduction Physics-Informed Neural Network (PINN) has achieved great success in scientific computing since 2017.

330 Jan 7, 2023

Physics-informed convolutional-recurrent neural networks for solving spatiotemporal PDEs

PhyCRNet Physics-informed convolutional-recurrent neural networks for solving spatiotemporal PDEs Paper link: [ArXiv] By: Pu Ren, Chengping Rao, Yang

11 Aug 23, 2022

PINN(s): Physics-Informed Neural Network(s) for von Karman vortex street

PINN(s): Physics-Informed Neural Network(s) for von Karman vortex street This is

2 Apr 18, 2022

A python implementation of Physics-informed Spline Learning for nonlinear dynamics discovery

PiSL A python implementation of Physics-informed Spline Learning for nonlinear dynamics discovery. Sun, F., Liu, Y. and Sun, H., 2021. Physics-informe

8 Jul 13, 2022

Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary Differential Equations

ODE GAN (Prototype) in PyTorch Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary

15 Feb 10, 2022

Leibniz is a python package which provide facilities to express learnable partial differential equations with PyTorch

Beijing ColorfulClouds Technology Co.,Ltd.

16 Aug 7, 2022

[CVPR 2021] Involution: Inverting the Inherence of Convolution for Visual Recognition, a brand new neural operator

involution Official implementation of a neural operator as described in Involution: Inverting the Inherence of Convolution for Visual Recognition (CVP

1.3k Dec 28, 2022

DI-HPC is an acceleration operator component for general algorithm modules in reinforcement learning algorithms

DI-HPC: Decision Intelligence - High Performance Computation DI-HPC is an acceleration operator component for general algorithm modules in reinforceme

185 Dec 29, 2022

Code for "Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations"

Infinitely Deep Bayesian Neural Networks with SDEs This library contains JAX and Pytorch implementations of neural ODEs and Bayesian layers for stocha

95 Nov 26, 2021

Supplementary code for the paper "Meta-Solver for Neural Ordinary Differential Equations" https://arxiv.org/abs/2103.08561

Meta-Solver for Neural Ordinary Differential Equations Towards robust neural ODEs using parametrized solvers. Main idea Each Runge-Kutta (RK) solver w

25 Aug 12, 2021

Official implementation of "Learning Forward Dynamics Model and Informed Trajectory Sampler for Safe Quadruped Navigation" (RSS 2022)

Intro Official implementation of "Learning Forward Dynamics Model and Informed Trajectory Sampler for Safe Quadruped Navigation" Robotics:Science and

21 Dec 7, 2022