Partial derivatives and singular value
decomposition for the analysis and control of
time-dependent spatially distributed systems

W. Castaings1, S. Tarantola1, F-X Le Dimet2, D. Dartus3
1Joint Research Center/IPSC, Ispra, IT
2 Joseph Fourier University, Grenoble,FR
3Toulouse Fluid Mechanics Institute, FR
william.castaings@jrc.it

The mathematical representation of natural and industrial systems frequently requires the use of time-dependent spatially distributed models. Although differential methods are intrinsically local, they are perfectly suited for some settings and may prove very informative. Suppose that the transformation between the independent variables x = (x1, x2, ..., xn) and the dependent variables y = (y1, y2, ..., ym) is represented by a function f : Rn  Rm. The response to be analysed can be a scalar but the study of a vectorial response tend to increase the understanding of the transformation. When function f is linearized at the given point of the input space (defining a trajectory in the model phase space), the partial derivatives are organized in the m × n Jacobian matrix J.
For instance, if the vector y accounts for the temporal evolution of an output variable of interest, each column of J is the result for all the time steps composing the response of an infinitesimal perturbation on one of the input parameters xi. Conversely, each line of J can represent the contribution of the different parameters xi (spatially distributed) on an aggregated response yj. While one can propose a physical interpretation for the lines and/or columns of the Jacobian matrix, a very interesting view angle is provided by its singular value decomposition (SVD). This factorization is widely used for the analysis of linear ill-posed problems [5] and its potential extrapolation to nonlinear systems is spreading in various disciplines (ex. [1], [3], [6]) . When applied to the Jacobian matrix of the transformation, it provides very relevant information for the analysis and control of the system under study. The first singular vectors in the input space describe the sub-space from the original control space maximizing the spread of the outputs, in the output space they identify the output variables really informative for the estimation of the model control variables. Together with the singular value spectrum, they are valuable elements to assess and enhance the sensitivity and identifiability of the independent variables. Going back to the illustration mentioned previously, if y accounts for the temporal evolution of an output variable and if a given spatially distributed parameter is composing the vector x, the parameters really influencing the response (spatial location) and the measurements really constraining the parameters (temporal location) are identified.

Although different techniques can be employed for the evaluation of the Jacobian matrix, algorithmic differentiation is very efficient in providing accurate derivatives. Tremendous advances have been made in this domain Griewank [4] and consequently the code based approach is facilitated by the advent of powerful automatic differentiation (AD) engines (see http://www.autodiff.org). The derivatives computed by means of algorithmic differentiation are accurate to the machine precision. Considering the computer code implementing the direct model as a concatenated sequence of instructions, algorithmic differentiation is based on a rigorous application of the chain rule, line by line. The application of the chain rule from the inputs to the outputs of the function is denoted as the forward mode of AD whereas the reverse mode operates from the outputs to the inputs. Because the computational cost of the reverse mode (discrete equivalent of the adjoint state method [2] from optimal control theory) is independent from the dimension of the input space, for vector valued response functions, it can be shown that when the ratio between the dimension of the input space and the dimension of the output space is greater than one, the reverse mode is more efficient in computing the Jacobian. 

Independently from the curse of dimensionality high-dimensional input spaces can be investigated while the use of sampling based approaches would imply a prohibitive computational cost. The described methodology is applied an catchment scale hydrological model representing the transformation of rainfall into runoff. Using the information provided by the SVD of the Jacobian matrix, it is shown that adjoint sensitivity analysis provides an extensive insight into the relation between model parameters and the hydrological response and enables the use of efficient gradient-based optimization techniques.

References

[1] Buizza, R. and Palmer, T. N.: The singular vector structure of the global atmospheric global circulation, Journal of Atmospheric Sciences, 52, 14341456, 1995.

[2]  Cacuci, D.: Sensitivity theory for nonlinear systems. I. Nonlinear functional analysis approach, Journal Mathematical Physics, 22(12), 2794–2802, 1981.

[3]  Clément, F., Khvoenkova, N., Cartalade, A., and Montarnal, P.: Analyse de sensibilit et estimation de paramtres de transport pour une quation de diffusion, approche par tat adjoint, Tech. Rep. RR-5132, INRIA-Rocquencourt, Projet ESTIME, 2004.

[4]  Griewank A. (2000). Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. Number 19 in Frontiers in Appl. Math. Philadelphia, PA: SIAM.

[5]  Hansen P. (1998). Rank-Deficient and Discrete Ill-Posed Problems. Numerical Aspects of Linear Inversion.
Philadelphia : SIAM.

[6]  Li, Z., Navon, I., and Hussaini, M. Y.: Analysis of the singular vectors of the full-physics Florida State University Global Spectral Model, Tellus, 57A, 2005.