Institute of Medical Biometry and Statistics (IMBI)

This project is completed and this page is archived. Last change on this page was 2008.

Model selection uncertainty and its influence on statistical prognostic and diagnostic models

Funding: funded by DFG up to December 2004

Description

Prognostic and diagnostic models usually are based on a model selection process trying to identify the most important factors influencing the prognosis or diagnosis. The selected models are the basis for predictions, classification rules and diagnostic tests. Although in practice one single "best" model is worked with, there usually is considerable uncertainty about which factors are really relevant and which model is most appropriate. Such model selection ucertainty can be accounted for by taking a weighted average of estimates from different possible models or by using the uncertainty attached to the models as a criterion for model selection. Although the main objective of these approaches is prediction, it must be pointed out that in medicine the interpretation of single effects also is important. Furthermore the models need to be transferable to other situations (other hospitals, a later time period, etc.).
Based on bootstrap resampling we proposed a bootstrap model averaging approach. By means of simulation studies we examine the properties of this method. For specific examples we compare our approach to "Bayesian model averaging". The main goal of this project is to assess whether incorporating model selection uncertainty by means of model averaging can improve prediction and if it leads to a more realistic assessment of their performance. Finally we aim to give recommendations about how to deal with model selection uncertainty in prognostic and diagnostic models.

Publications

• Sauerbrei W, Holländer N, Buchholz A: Investigation about a screening step in model selection. Statistics and Computing, 2008; 18: 195-208.
• Buchholz A, Holländer N, Sauerbrei W: On properties of predictors derived with a two-step bootstrap model averaging approach - A simulation study in the linear regression model. Computational Statistics and Data Analysis, 2008; 52: 2778-2793.
• Holländer N, Augustin N, Sauerbrei W: Investigation on the improvement of prediction by bootstrap model averaging. Methods of Information in Medicine, 2006; 45:44-50.
• Augustin N, Sauerbrei W, Schumacher M: The practical utility of incorporating model selection uncertainty into prognostic models for survival data. Statistical Modelling, 2005; 5: 95-118.