Entropy-based sensitivity analysis in health risk assessment

R. Landri1,2,*, G. Ducharme1, R. Bonnard2
1Montpellier University, France; 2INERIS, France
remi.landri@math.univ-montp2.fr

For realistic decision-making in risk management, the reliability and quality of exposure and risk model outputs, which depend on uncertainty, must be known. Therefore, sensitivity analysis is a central point to study the problem of how the output  is sensitive to the uncertainty of the input variables . Usually, output uncertainty is described in terms of variance. Variance is popular in sensitivity analysis because of its simplicity and its historical development; statisticians also use it as a reference measure of the dispersion.

Other measures may be used to characterize the uncertainty of a model output related to an input variable. Krykacz-Haussmann [1] has criticized the use of variance as a measure of output uncertainty and proposed to use the entropy. Entropy is an information criterion which measures the amount of uncertainty or information content that is implied by a probability distribution. It has the advantage of depending on many more parameters than just the second moment which allows only to measure a dispersion around the mean. Moreover, the conditional variance ,  can be larger than  while the entropy  verifies . Shannon’s entropy of a random vector  of  with probability density function, is defined as:

Note that the entropy of continuous random variables is not invariant under a continuous invertible transformation of variables.

To anticipate the impact of input vector variations to the output in terms of entropy, the following result [2] is very useful when the dimensions of the two vectors are the same: Let  and  be two random vectors of  variables, and  a continuously differentiable transformation such that , then:
,
where  is the probability density of  and  is the Jacobian of the inverse transformation . Unfortunately, such a result is not yet available for vectors of different dimensions: . Furthermore, unlike in the univariate case, many entropy expressions aren’t calculable in multivariate situation.

Comparatively to the variance for which the expression  is used, the “mutual information”  between two random vectors  and , is an invariant (under one-to-one transformations), symmetric and natural measure of the dependence between  and , and corresponds to:
.
Sensitivity indices based on mutual information seems to be as much attractive as variance-based sensitivity analysis one, to obtain “importance measure” on input variables.

There is no universal relationship between entropy and variance “orderings of distribution”. Mukherjee and Ratnaparkhi [3] presented some relationships between the two for some particular univariate distributions. Writing the entropy in terms of variance, Ebrahimi and al. [4] find the orderings across some families of univariate distributions. Also, “” implies that “” but the converse may not hold. That means entropy measures could exist even if variance ones are infinite or not defined. For example, the variance of the Cauchy distribution, or the T-Student distribution one for some parameters, does not exist, while their entropy always does. The entropy  is a dimensionless scalar measure of a distribution; consequently in the multivariate case, entropy rankings can be easily obtained. On the contrary, the variance  approach leads to a matrix of order  and variance rankings are based on arbitrary measures, such as the determinant or the trace of the variance-covariance matrix. We show that entropy depends on much more information about a vector of random variables than its variance.
A copula  is a multivariate probability density function defined on the unit cube  (each marginal law is a uniform  dependent to each other). The copula approach has the advantage of characterizing, splitting and modelling the whole dependence structure between the  random variables , whatever the marginal distributions.

In the bivariate case, we have examined entropy (and mutual information) and variance orderings of parametric copulas families  with respect to some functions of margins, such as sum and product. We present and discuss some analytical formulas and graphical relationships between variance and entropy of several bivariate copulas, such as Farlie-Gumbel-Morgenstern, Gamma-Exponential or Dirichlet. We also compare joint, conditional and marginal entropies and variances using the following property:
.

As a practical application, the U.S. Environmental Protection Agency’s Integrated Exposure Uptake Bio Kinetic (IEUBK) model for lead in children allows to estimate children’s blood lead levels by integrating exposure from lead in air, food, water, soil, dust and other sources with pharmacokinetic modelling. Unfortunately, the model doesn’t assume any probability distribution for the input variables. Thus, we have coded the model to carry out Monte Carlo simulations on input variables corresponding to children’s disparity in order to obtain the blood lead concentration’s distribution which conveys the output variability. Empirical simulation results and entropy-based sensitivity analysis appear to be helpful in focalising on the major aspects of the IEUBK model to take into account the input distributions. Our results are compared to variance-based sensitivity analysis in terms of “expected reduction” of the uncertainty.

References

[1] Krykacz-Haussmann B: Epistemic sensitivity analysis based on the concept of entropy. In: Proceedings of SAMO2001 (eds P. Prado and R. Bolado) Madrid, CIEMAT, 2001, pp. 31-35.

[2] Pardo L: Statistical Inference Based on Divergence Measures. Chapman&Hall, London, 2006.

[3] Mukherjee D, Ratnaparkhi MV: On the functional relationship between entropy and variance with related applications. Communications in Statistics-Theory and Methods 15, 291-311(1986)

[4] Ebrahimi N, Maasoumi E, and Soofi ES: Ordering univariate distributions by entropy and variance. Journal of Econometrics 90, 317-336(1999)