Multi-fidelity modeling enables the seamless fusion of information from a collection of heterogeneous sources of variable accuracy and cost (e.g., noisy experimental data, computer simulations, empirical models, etc.). By learning to exploit the cross-correlation structure between these sources, one can construct predictive surrogate models that can dramatically reduce the compute time to solution. The impact of multi-fidelity modeling has already been recognized in our project on shape optimization of super-cavitating hydrofoils. The application involves the design optimization of an ultrafast marine vehicle for special naval operations.

Engineering Application: The application involves the design optimization of an ultrafast marine vehicle for special naval operations.

This problem inherits numerous and laborious challenges including the modeling of complex turbulent and multi-phase fluid flows, the solution of high-dimensional optimization problems, and the assessment of risk due to uncertainty in environmental and operational conditions. Here, the introduction of multi-fidelity modeling enables us to combine high-fidelity turbulent multi-phase flow simulations, experimental data, and simplified low-fidelity models (e.g., potential flow simulations), and efficiently tackle this large-scale optimization task that currently seems daunting to any other approach.

Deep Multi-fidelity Gaussian Processes

A simple way to explain the main idea of this work is to consider the following structure:


The high fidelity system is modeled by and the low fidelity one by . We use to denote a Gaussian Process. This approach can use any deterministic parametric data transformation . However, we focus on multi-layer neural networks

where each layer of the network performs the transformation

with being the transfer function, the weights, and the bias of the layer. We use to denote the parameters of the neural network. Moreover, and denote the hyper-parameters of the covariance functions and , respectively. The parameters of the model are therefore given by


The Deep Multi-fidelity Gaussian Process structure can be equivalently written in the following compact form of a multivariate Gaussian Process

with , and . This can be used to obtain the predictive distribution

of the surrogate model for the high fidelity system at a new test point . Note that the terms and model the correlation between the high-fidelity and the low-fidelity data and therefore are of paramount importance. The key role played by is already well-known in the literature. Along the same lines one can easily observe the effectiveness of learning the transformation function $h(x)$ jointly from the low fidelity and high fidelity data.

We obtain the following joint density:

where , , and . From this, we conclude that



The negative log marginal likelihood is given by \begin{eqnarray}\label{Likelihood} \mathcal{L}(\theta) = \frac12 \mathbf{f}^T K^{-1}\mathbf{f} + \frac12 \log \left| K \right| + \frac{n_1 + n_2}{2}\log 2\pi, \end{eqnarray} where

The negative log marginal likelihood along with its Gradient can be used to estimate the parameters .

Deep Multi-fidelity Gaussian Processes predictive mean and two standard deviations.


We devised a surrogate model that is capable of capturing general discontinuous correlation structures between the low- and high-fidelity data generating processes. The model’s efficiency in handling discontinuities was demonstrated using benchmark problems. Essentially, the discontinuity is captured by the neural network. The abundance of low-fidelity data allows us to train the network accurately. We therefore need very few observations of the high-fidelity data generating process.


This work was supported by the DARPA project on Scalable Framework for Hierarchical Design and Planning under Uncertainty with Application to Marine Vehicles (N66001-15-2-4055).


  title={Deep Multi-fidelity Gaussian Processes},
  author={Raissi, Maziar and Karniadakis, George},
  journal={arXiv preprint arXiv:1604.07484},