Risk analysis models almost invariably involve some element of subjective estimation. It is usually impossible to obtain data from which to accurately determine the uncertainty of all of the variables within the model for a number of reasons:
The data have simply never been collected in the past
The data are too expensive to obtain
Past data are no longer relevant (new technology, changes in political or commercial environment, etc.)
The data are sparse, requiring expert opinion to fill the data gaps
The area being modeled is new
The uncertainty in subjective estimates has two components: the inherent randomness of the variable itself and the uncertainty arising from the expert's lack of knowledge of the model parameter. In a risk analysis model, these uncertainties may or may not be distinguished but both types of uncertainty should at least be accounted for in a model. The variability is best included by assuming some sort of stochastic model and the uncertainty is then included in the uncertainty distributions for the model parameters.
When insufficient data are available to completely specify the uncertainty of a variable, one or more experts will usually be consulted to provide their opinion of the variable's uncertainty. This section offers guidelines for the analyst to model the experts' opinions as accurately as possible.
We recommend that you first review the discussion on potential sources of bias and error that the analyst will be faced with when collecting subjective estimates. Then review the discussion of techniques used in the modeling of probabilistic estimates and particularly the use of various types of distributions.