In-context operator learning with data prompts for differential equation problems (2023-04-17T00:00:00.000000Z)

TL;DR

Numerical results demonstrate the efficacy of ICON in solving various types of differential equation problems and generalizing to operators beyond the training distribution and has implications for artificial general intelligence in physical systems.

Abstract

Significance This paper presents In-Context Operator Networks (ICON), a neural network approach that can learn new operators from prompted data during the inference stage without requiring any weight updates. Unlike existing methods that are limited to approximating specific equation solutions or operators and necessitate retraining for new problems, ICON trains a single neural network as an operator learner, eliminating the need for retraining or fine-tuning when encountering different problems. Numerical results demonstrate the efficacy of ICON in solving various types of differential equation problems and generalizing to operators beyond the training distribution. The proposed approach draws inspiration from successful in-context learning techniques used in natural language processing and has implications for artificial general intelligence in physical systems.

Authors

Liu Yang

2 papers

Siting Liu

1 papers

Tingwei Meng

1 papers

TL;DR

Abstract

Authors

References45 items

Multi-fidelity prediction of fluid flow and temperature field based on transfer learning using Fourier Neural Operator

Reliable extrapolation of deep neural operators informed by physics or sparse observations

Transfer Learning Enhanced DeepONet for Long-Time Prediction of Evolution Equations

Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering

Explaining the physics of transfer learning in data-driven turbulence modeling

Domain adaptation based transfer learning approach for solving PDEs on complex geometries

Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios

Deep transfer operator learning for partial differential equations under conditional shift

Chain of Thought Prompting Elicits Reasoning in Large Language Models

Learning Operators with Coupled Attention

Transfer learning for deep neural network-based partial differential equations solving

Meta-Auto-Decoder for Solving Parametric Partial Differential Equations

Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing

Learning Functional Priors and Posteriors from Data and Physics

A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics

Physics-informed machine learning

Mosaic flows: A transferable deep learning framework for solving PDEs on unseen domains

Learning the solution operator of parametric partial differential equations with physics-informed DeepONets

Machine learning–accelerated computational fluid dynamics

Meta-MgNet: Meta Multigrid Networks for Solving Parameterized Partial Differential Equations

Fourier Neural Operator for Parametric Partial Differential Equations

Analysis of three-dimensional potential problems in non-homogeneous media with physics-informed deep collocation method using material transfer learning and sensitivity analysis

Language Models are Few-Shot Learners

End-to-End Object Detection with Transformers

Transfer learning based multi-fidelity physics informed deep neural network

Model Reduction and Neural Networks for Parametric PDEs

Alternating the population and control neural networks to solve high-dimensional stochastic mean-field games

A machine learning framework for solving high-dimensional mean field game and mean field control problems

Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

Weak Adversarial Networks for High-dimensional Partial Differential Equations

Transfer learning enhanced physics informed neural network for phase-field modeling of fracture

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

Bayesian Deep Convolutional Encoder-Decoder Networks for Surrogate Modeling and Uncertainty Quantification

PDE-Net: Learning PDEs from Data

The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems

DGM: A deep learning algorithm for solving partial differential equations

Solving high-dimensional partial differential equations using deep learning

Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations

Attention is All you Need

Discovering governing equations from data by sparse identification of nonlinear dynamical systems

Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems

Approximation capability to functions of several variables, nonlinear functionals, and operators by radial basis function neural networks

Neural Operator: Learning Maps Between Function Spaces With Applications to PDEs

In-context operator learning with data prompts for differential equation problems

Language Models are Unsupervised Multitask Learners

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names