Introducing the Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations. I: Mathematical Framework

This work introduces the mathematical framework of the novel “First-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations” (1st-FASAM-NODE). The 1st-FASAM-NODE methodology produces and computes most efficiently the exact expressions of all of the first-order sensitivities of NODE-decoder responses with respect to the parameters underlying the NODE’s decoder, hidden layers, and encoder, after having optimized the NODE-net to represent the physical system under consideration. Building on the 1st-FASAM-NODE, this work subsequently introduces the mathematical framework of the novel “Second-Order Features Adjoint Sensitivity Analysis Methodology for Neural Ordinary Differential Equations (2nd-FASAM-NODE).” The 2nd-FASAM-NODE methodology yields and computes most efficiently the exact expressions of the second-order sensitivities of NODE decoder-responses with respect to the NODE parameters. Since the physical system modeled by the NODE-net necessarily comprises imprecisely known parameters that stem from measurements and/or computations subject to uncertainties, the availability of the first-and second-order sensitivities of decoder responses to the parameters underlying the NODE-net are essential for performing sensitivity analysis and quantifying the uncertainties induced in the NODE-decoder responses by uncertainties in the underlying uncertain NODE-parameters.