A model validation method with Bootstrap approach and Bayes estimation for small sample

Abstract

Generally, model validation is mainly based on statistical analysis. However, when the sample size of real system output is small, it is difficult to obtain accurate validation results with classical statistics theory. In such a situation, a model validation method based on improved Bootstrap approach and Bayes estimation is provided. First, Bootstrap method is used to enlarge observed samples size and obtain Bayes prior distribution information. Then, Bayes theory which combines prior information and small sample data is used to estimate the statistical characteristics of observed samples. Finally, single-sample hypothesis testing is used to evaluate the credibility of simulation model. Furthermore, an improved Bootstrap method is proposed, which raises the accuracy of parameter estimation and extends bootstrap samples range beyond the original data. The numerical experiment results reveal the effectiveness of validation method and improved Bootstrap method.

Model Validation | Small Sample | Improved Bootstrap Method | Bayes Estimation
 

References

1. Ahmed F., Mohamed A., Jaeyoung L., Naveen E., 2017. Application of Bayesian informative priors to enhance the transferability. Journal of Safety Research, 62, 155-161
2. Bunouf P., Lecoutre B., 2006. Bayesian priors in sequential binomial design. Comptes Rendus
   Mathematique, 343, 339-334.
3. Chihara L., Hesterberg T., 2011. Mathematical Statistics with Resampling and R. Hoboken, NJ,
   USA: Wiley.
4. Dai Z.H., Wang Z., Jiao Y., 2014. Bayes monte-carlo assessment method of protection systems
   reliability based on small failure sample data. IEEE Transactions on Power Delivery, 29(4),
   1841-1848.
5. Efron B., 1979. Bootstrap methods: another look at the jackknife. Annals of Statistics, 7(1), 1-26.
6. Heydari S., Miranda-Moreno L., Lord D., Fu L., 2014. Bayesian methodology to estimate and update safety performance functions under limited data conditions: A sensitivity analysis. Accident Analysis and Prevention, 64, 41–51.
7. Kyselý J., 2010. Coverage probability of Bootstrap confidence intervals in heavy tailed frequency models, with application to precipitation data. Theoretical and Applied Climatology, 101(3-4),345–361.
8. Li C.Z., Mahadevan S., 2016. Role of calibration, validation, and relevance in multi-level uncertainty integration. Reliability Engineering & System Safety, 148, 32-43.
9. Oberkampf W.L., Barone M.F., 2006. Measures of agreement between computation and experiment: validation metrics. Journal of Computational Physics, 217(1), 5–36.
10. Sargent R.G., 2013. Verification and validation of simulation models. Journal of Simulation, 7(1),
    12-24.
11. Samart K., Jansakul N., Chongcheawchamnan M., 2017. Exact bootstrap confidence intervals for regression coefficients in small samples. Communication in Statistics-Simulation and Computation. 1-7 (electronically published).
12. Tang X.M., 2002. Validation method of simulation model under small sample size. Journal of System Simulation, 14(10), 1263-1266.
13. Xiao Z.C., Gao H.M., Ding T., Han J.F., 2009. The application of self-help method in mean estimation of small sample data. Journal of Naval Aeronautical Engineering Institute, 24(5), 563-567.
14. Yalcinkaya M., Birgoren B., 2017. Confidence interval estimation of Weibull lower percentiles in small samples via Bayesian inference. Journal of the European Ceramic Society, 37, 2983-2990.
15. Zhang S.Y., Feng W.S., 2009. Study of sampled data creation for norm distribution on Bootstrap
    method. Journal of Academy of Equipment Command & Technology, 20(2), 97-100.