| Title | Validating Simulation Models |
| Speaker | Dr. Robert Marks |
| Chair | Shu-Heng Chen |
Abstract
We can think of two types of simulation models: Demonstration models, essentially existence proofs for phenomena of interest, and Descriptive models, that attempt to track dynamic historical phenomena. Both types require verification. Descriptive models require validation against historical data as well.
Model validation is an attempt to assure the reader that the model is “good” at being able to generate the observed data. The observed data contain information, and the models developed (from our theoretical understanding of the underlying processes generating the observed data) are attempts to express these data in as compact a form as possible. All simulation models are existence proofs; necessity is harder to establish.
The ultimate goal of modeling is to derive a model that produces a set of output data identical to the historical. Since models are only approximations of reality, this idealized goal of a complete and accurate model is unattainable, and often undesirable because of over-fitting. With several contending models, validation might be able to point the researcher to the “best” model, in the sense that it loses least information.
The tutorial will introduce a formal framework for validation. It will discuss how simulations might establish necessity, as well as sufficiency. The tutorial will compare two uses of simulation: to analyze systems and to design systems, the former the traditional social sciences use, and the latter the engineering use. Given that simulation of systems can result in several streams of output data, we shall discuss techniques of comparing sets of time series in order to validate such models against sets of historical data.
Biography
Dr. Robert Marks is an Emeritus Professor of Economics at the University of New South Wales, and a Professorial Fellow at the University of Melbourne. He was a foundation faculty member at the Australian Graduate School of Management. He published the first application of the Genetic Algorithm in game theory, and was for 13 years the Editor-in-Chief of the Australian Journal of Management. He has published over 171 papers and reports, including pioneering papers on computer modeling and validation in the social sciences.
