A Lagrangian dual-based theory-guided deep neural network

Rong, Miao1,2; Zhang, Dongxiao3; Wang, Nanzhe4

2022-04-01

10.1007/s40747-022-00738-1

Complex & Intelligent Systems   影响因子和分区

2199-4536

2198-6053

The theory-guided neural network (TgNN) is a kind of method which improves the effectiveness and efficiency of neural network architectures by incorporating scientific knowledge or physical information. Despite its great success, the theory-guided (deep) neural network possesses certain limits when maintaining a tradeoff between training data and domain knowledge during the training process. In this paper, the Lagrangian dual-based TgNN (TgNN-LD) is proposed to improve the effectiveness of the training process. We convert the original loss function into a constrained form with several items, in which partial differential equations (PDEs), engineering controls (ECs), and expert knowledge (EK) are regarded as constraints, with one Lagrangian variable per constraint. These Lagrangian variables are incorporated to achieve an equitable trade-off between observation data and corresponding constraints, to improve prediction accuracy and training efficiency. To investigate the performance of the proposed method, the original TgNN model with a set of optimized weight values adjusted by ad-hoc procedures is compared on a subsurface flow problem, with their L2 error, R square (R2), and computational time being analyzed. Experimental results demonstrate the superiority of the Lagrangian dual-based TgNN.

Theory-guided neural network Lagrangian dual Weights adjustment Trade-off

SCI

英语

通讯

National Natural Science Foundation of China[62103255]

Computer Science

Computer Science, Artificial Intelligence

WOS:000787125400004

SPRINGER HEIDELBERG

Web of Science

1.Peng Cheng Lab, Dept Math & Theories, Shenzhen 518000, Peoples R China
2.Shanghai Univ, Sch Mechatron Engn & Automat, Shanghai 200444, Peoples R China
3.Southern Univ Sci & Technol, Sch Environm Sci & Engn, Shenzhen 518055, Peoples R China
4.Peking Univ, Coll Engn, Beijing 100871, Peoples R China

Rong, Miao,Zhang, Dongxiao,Wang, Nanzhe. A Lagrangian dual-based theory-guided deep neural network[J]. Complex & Intelligent Systems,2022.

Rong, Miao,Zhang, Dongxiao,&Wang, Nanzhe.(2022).A Lagrangian dual-based theory-guided deep neural network.Complex & Intelligent Systems.

Rong, Miao,et al."A Lagrangian dual-based theory-guided deep neural network".Complex & Intelligent Systems (2022).