A General Framework for Validation of Computer Models and Simulations

James C. Cavendish, General Motors Research and Development Center

Computer simulations and math models are used in industry to design, understand and control various physical systems and processes and to predict the performance of these systems and processes under various operating conditions. The two main approaches to establishing credible confidence statements about the predictive capabilities of computer models are verification and validation. Verification is the process of determining that a computer model (that is, the operational computer program or "code") is a correct (i.e. bug-free) and accurate implementation of a conceptual model of a physical system or process. Model validation is the process of determining the degree to which output from a computerized model accurately represents reality. In this talk we present a framework that enables the implementation of the validation process.

This framework is being developed as part of a Research Agreement between the National Institute of Statistical Sciences (NISS) and General Motors. Underlying the proposed validation framework is a systematic six-step procedure based upon a Bayesian statistical methodology. The Bayesian methodology is particularly suited to treating the major issues associated with the validation process: quantifying multiple sources of error and uncertainty in math models; combining multiple sources of information; and updating validation assessments as new information is acquired. Moreover, it allows inferential statements to be made about predictive error associated with model predictions in untested situations (that is, prediction beyond the validation comparisons between calculations and experimental outcomes). The framework is described with the help of a test bed model that provides context for each of the six steps in the proposed validation process.

See related publication: M.J. Bayarri, J.O. Berger, D. Higdon, M. Kennedy, A. Kottas, R. Paulo, J. Sacks, J.A. Cafeo, J.C. Cavendish, C-H lin, J. Tu (2002), "A Framework for Validation of Computer Models", Technical Report 128, NISS www.niss.org/downloadabletechreports.html.

Return to Program