Speeding up Collocation-based Robust Variational Physics Informed Neural Networks with Singular Value Decompositions

Abstract

Physics Informed Neural Networks (PINNs) solve time-dependent Partial Differential Equations (PDEs) by minimizing the residual, initial, and boundary conditions loss functions. The loss functions in PINN are generally not robust. The decrease in the loss function may not correspond to the decrease by the true error (the norm difference between the numerical and exact solution). Following the idea of Robust Variational PINN (RVPINN), we define the robust loss function by employing the weak residuals and the inverse of the Gram matrix as the discrete representative of the Riesz residual. To avoid the expensive integration of the weak residuals, we introduce the collocation method, where we use discrete weak formulations with integration replaced by summations at the collocation points and test functions replaced by Kronecker deltas. With this setup, we derive a sparse Gram matrix as the H¹₀ discrete inner product of the Kronecker delta test functions. The inverse of this Gram matrix can make the loss function robust. Furthermore, to accelerate the training process, we perform a truncated Singular Value Decomposition (SVD) of the inverse of the Gram matrix and use fast matrix-vector multiplication for the reduced rank matrix. The presented strategy reduces the cost of precomputing (before training) from 𝒪(N³) (inversion of the Gram matrix) to 𝒪(N²k) (truncated SVD decomposition for k << N, and the cost of training from 𝒪(N²) to (Nk) in each iteration of training.