1. Proceedings
  2. I3M
  3. 2020
  4. MAS
  5. 10.46354/i3m.2020.mas.005

Effective use of verification and validation resources

  • a Steven Darcy , 
  • b Daniel Maxwell , 
  • c Andrew Loerch 
  • a,c George Mason University, 4400 University Drive MSN 4A6, Fairfax, VA, 22030, USA
  • b KaDSci LLC, 031 University Dr., Suite 100, Fairfax, VA, 22030, USA
Cite as
Darcy S., Maxwell D., Loerch A. (2020). Effective use of verification and validation resources. Proceedings of the 19th International Conference on Modeling & Applied Simulation (MAS 2020), pp. 34-43. DOI: https://doi.org/10.46354/i3m.2020.mas.005

Abstract

A model has force if it can positively and constructively influence decision makers. To a large extent, the force a model has on a decision maker is determined by the confidence they have in the model. Ideally that confidence is a consequence of the model performing well in a high-quality verification and validation process. This paper presents a generalization of the Line of Sight Evidential Reasoning and Analysis methodology. This methodology is used to construct a Bayesian network which codifies our belief about how specific verification and validation activities change a model’s force on a decision maker. Using the Bayesian network we rank verification and validation activities based on how much impact they have on the model’s force. We then show how the ranking of verification and validation activities can change based on the intended use of the model. In other words, the Generalized Line of Sight Evidential Reasoning and Analysis methodology provides a quantitative basis for allocating V&V resources and this allocation may be different for different decision makers or model uses.

References

  1. VV&A, 12/9/2009: DoD Instruction 5000.61. Available online at http://www.dtic.mil/whs/directives/corres/pdf/500
    061p.pdf
  2. Anamaria Berea; Daniel Maxwell (2017): A Bayesian Model for Investment Decisions In Early Ventures
  3. C. E. Shannon (1948): A Mathematical Theory of Communication. In Bell Labs Technical Journal 27 (3), pp. 379–423.
  4. Darwiche, Adnan (2014): Modeling and reasoning with Bayesian networks. New York, N.Y.: Cambridge University Press.
  5. Norsys: Netica. Available online at https://www.norsys.com/netica.html.
  6. Sargent, Robert G. (2010): Verification and validation of simulation models. In : WSC 2010, Winter Simulation Conference. Baltimore, MD, USA, pp. 166–183.
  7. Schum, D.: The Evidential Foundations of Probabilistic Reasoning.
  8. Waltz, Edward (2003): Knowledge management in the intelligence enterprise. London: Artech House.
  9. William Bunting (2012): Enterprise Line of Sight Analysis Using Evidential Reasoning. George Mason
    University, Fairfax, VA.