Multi-Objective Loss Balancing for Physics-Informed Deep Learning (2021-10-19T00:00:00.000000Z)

TL;DR

This work proposes a novel self-adaptive loss balancing scheme for PINNs named ReLoBRaLo (Relative Loss Balancing with Random Lookback), and shows that ReLoBRaLo is able to consistently outperform the baseline of existing scaling methods in terms of accuracy, while also inducing significantly less computational overhead.

Authors

Rafael Bischof

1 papers

M. Kraus

1 papers

TL;DR

Authors

References83 items

Failure-informed adaptive sampling for PINNs

A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks

A novel meta-learning initialization method for physics-informed neural networks

A Physics Informed Neural Network for Time-Dependent Nonlinear and Higher Order Partial Differential Equations

A physics-informed deep learning framework for inversion and surrogate modeling in solid mechanics

Parallel Physics-Informed Neural Networks via Domain Decomposition

Self-adaptive loss balanced Physics-informed neural networks

Sobolev training of thermodynamic-informed neural networks for interpretable elasto-plasticity models with level set hardening

A Deep Collocation Method for the Bending Analysis of Kirchhoff Plate

Heat Transfer Prediction With Unknown Thermal Boundary Conditions Using Physics-Informed Neural Networks

Artificial intelligence for structural glass engineering applications — overview, case studies and future potentials

A fast and accurate physics-informed neural network reduced order model with shallow masked autoencoder

Self-Adaptive Physics-Informed Neural Networks using a Soft Attention Mechanism

When and why PINNs fail to train: A neural tangent kernel perspective

Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks

CoPhy-PGNN: Learning Physics-guided Neural Networks with Competing Loss Functions for Solving Eigenvalue Problems

Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks

Implicit Neural Representations with Periodic Activation Functions

Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems

Extended Physics-informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition based Deep Learning Framework for Nonlinear Partial Differential Equations

Transfer learning based multi-fidelity physics informed deep neural network

Machine learning for structural health monitoring: challenges and opportunities

Accelerating Physics-Informed Neural Network Training with Prior Dictionaries

On the Convergence of Physics Informed Neural Networks for Linear Second-Order Elliptic and Parabolic Type PDEs

hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition

Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations

Understanding and mitigating gradient pathologies in physics-informed neural networks

Geometric deep learning for computational mechanics Part I: Anisotropic Hyperelasticity

SoftAdapt: Techniques for Adaptive Loss Weighting of Neural Networks with Multi-Part Loss Functions

Grid Search, Random Search, Genetic Algorithm: A Big Comparison for NAS

PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs

Meta-Learning with Implicit Gradients

DeepXDE: A Deep Learning Library for Solving Differential Equations

Transfer learning enhanced physics informed neural network for phase-field modeling of fracture

Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data

Adaptive activation functions accelerate convergence in deep and physics-informed neural networks

Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations

Machine Learning Techniques for the Material Parameter Identification of Laminated Glass in the Intact and Post-Fracture State

Multi-Task Learning as Multi-Objective Optimization

Eurocode 2: Design of concrete structures

A proof that artificial neural networks overcome the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations

Probabilistic Model-Agnostic Meta-Learning

A systematic literature review of Burgers’ equation with recent advances

Group Normalization

On First-Order Meta-Learning Algorithms

Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations

Theory of Deep Learning IIb: Optimization Properties of SGD

Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations

GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks

Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm

DGM: A deep learning algorithm for solving partial differential equations

Attention is All you Need

Sobolev Training for Neural Networks

Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A review

Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality

Adam: A Method for Stochastic Optimization

Practical Bayesian Optimization of Machine Learning Algorithms

Understanding the difficulty of training deep feedforward neural networks

Cross-disciplinary perspectives on meta-learning for algorithm selection

Numerical Treatment of Partial Differential Equations

Multi-objective meta-heuristics: An overview of the current state-of-the-art

Neural-network methods for boundary value problems with irregular boundaries

Use of a self-adaptive penalty approach for engineering optimization problems

Multitask Learning

Artificial neural networks for solving ordinary and partial differential equations

Partial Differential Equations of Physics

Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks

Sobolev Training for the Neural Network Solutions of PDEs

Efficient Multi-Objective Optimization for Deep Learning

Burgers equation

Online Learning: the Stochastic Gradient Descent Family of Algorithms

Finite Element Method Linear Static And Dynamic Finite Element Analysis

Chapter 17 – Design Optimization

The application of Bayesian methods for seeking the extremum

Bayesian Optimization: Open source constrained global optimization tool for Python, 2014