To learn more about EpiX Analytics' work, please visit our modeling applications, white papers, and training schedule.

Page tree

 

The simulation software package can perform a crude sensitivity analysis that is often used to identify the key input variables, as a step after performing an initial sensitivity analysis on these key variables. It achieves this by computing rank order correlation coefficients from the data that have been generated from inputting distributions and data calculated for the selected output. Built into this operation are two important assumptions:

 

  1. All the tested input parameters have either a purely positive or negative statistical correlation with the output; and

  2. Each uncertain variable is modeled with a single distribution

 

Assumption 1. is rarely invalid, but would be incorrect if the output value was at a maximum or minimum for an input value somewhere in the middle of its range, e.g.:

 

 

Assumption 2. is very often incorrect. For example, the impact of a risk event might be modeled as:

 

=Binomial(20%, 1)*Triangular(10,20,50)

 

Performing the standard sensitivity analysis in the simulation software package will evaluate the effect of the Binomial and the Triangular distributions separately, so the measured effect on the output will be divided between these two distributions. There is a trick to get around this problem, however. You can "fool' the simulation software package into performing a crude sensitivity analysis on a combined variable. You make the cell generating the random value for that variable the input parameter for a Discrete uniform distribution. Since there is only this one input parameter, the Discrete uniform distribution will return whatever value is being generated for that input variable.

 

Assumption 2 also means that this method of sensitivity analysis is invalid for a variable that is modeled over a series of cells, like a time series of exchange rates or sales volumes. The automated analysis will evaluate the sensitivity of the output to each distribution separately. You can still evaluate the sensitivity of a time series by running two simulations: one with all the distributions simulating random values, and another with the distributions of the time series frozen ("locked") to their expected value. For some simulation software packages such as @Risk, there are special features to lock the variable (to learn how to lock variables in @Risk, see the following models: RiskLock1 & RiskLock2). If the distributions vary significantly, the variable time series is important.

The links to the RiskLock software specific models are provided here:

  Risklock1

 Risklock2 

Note that you can lock the assumptions with the "Freeze" feature in the simulation software package. You need to select "Cell" >> "Freeze Assumptions" and then select which assumptions you temporarily want to freeze.


Two sensitivity scales

The simulation software package may determine sensitivity through different sensitivity analysis methods. The sensitivity analysis method plots the variable against rank order correlation coefficient statistic which takes values from -1 (the output is wholly dependent on this input, but when the input is large, the output is small), through 0 (no influence) to +1 (the output is wholly dependent on this input, and when the input is large, the output is also large):

 

 

  1. Rank order correlation. This analysis replaces each collected value by its rank among other values generated for that input or output, and then calculates the Spearman's rank order correlation coefficient ρ between each input and the output. Since this is a non-parametric analysis, it is fairly robust where there are complex relationships between the inputs and output.

  2. Contribution To Variance. This option which you can select in the "Sensitivity Preferences dialog" in the simulation software package, lets you see the sensitivities as a percentage of the contribution to the variation of the outcome of interest. While it would not change the order of the items listed in the rank order correlation sensitivity chart, it's scale attempts to answer the question how much variation or uncertainty in the target forecast is due to assumption X. Note that the Contribution to Variance method is only an approximation and is not precisely a variance decomposition. It is calculated in the simulation software package by squaring the rank order correlation, and normalizing all to 100%.

  3. Stepwise least – squares regression between collected input distribution values and the selected output values. The assumption here is that there is a relationship between each input I and the output O (when all other inputs are held constant) of the form O = I*m + c where m, c are constants. That assumption is correct for additive and subtractive models, and will give very accurate results in those circumstances, but is otherwise less reliable and somewhat unpredictable. The r-squared statistic is then used as the measure of sensitivity in a Tornado chart.

Each simulation software package has its own specification. For example, in Crystal Ball, the Tornado chart can be displayed with sensitivity scales 1 & 2, while in @Risk, analyses methods 2 & 3 are used. Both methods in @Risk require that you have either selected the Collect All Distribution Samples option in Simulation Settings, or selected the Inputs Marked with Collect and marked some of the model’s distributions with the RiskCollect( ) function.

 

 

 

 


  • No labels