%0 Conference Paper
%B Proceedings of the 5th International Conference on Predictor Models in Software Engineering
%D 2009
%T Using uncertainty as a model selection and comparison criterion
%A Sarcia',Salvatore Alessandro
%A Basili, Victor R.
%A Cantone,Giovanni
%K accuracy
%K Bayesian prediction intervals
%K Calibration
%K cost estimation
%K cost model
%K model evaluation
%K model selection
%K prediction interval
%K Uncertainty
%X Over the last 25+ years, software estimation research has been searching for the best model for estimating variables of interest (e.g., cost, defects, and fault proneness). This research effort has not lead to a common agreement. One problem is that, they have been using accuracy as the basis for selection and comparison. But accuracy is not invariant; it depends on the test sample, the error measure, and the chosen error statistics (e.g., MMRE, PRED, Mean and Standard Deviation of error samples). Ideally, we would like an invariant criterion. In this paper, we show that uncertainty can be used as an invariant criterion to figure out which estimation model should be preferred over others. The majority of this work is empirically based, applying Bayesian prediction intervals to some COCOMO model variations with respect to a publicly available cost estimation data set coming from the PROMISE repository.
%B Proceedings of the 5th International Conference on Predictor Models in Software Engineering
%S PROMISE '09
%I ACM
%C New York, NY, USA
%P 18:1–18:9 - 18:1–18:9
%8 2009///
%@ 978-1-60558-634-2
%G eng
%U http://doi.acm.org/10.1145/1540438.1540464
%R 10.1145/1540438.1540464