We are frequently asked in courses we give how to determine how many iterations to run a Monte Carlo model for. It's a good question, but the answer depends on a few other things you need to work out first. In the "old' (i.e. maybe ten years ago) computers ran much more slowly, of course. The first Crystal Ball models, which ran on Lotus 1-2-3, would take hours to run a few hundred iterations on very simple models. Of course, there is a certain propensity to produce more detailed and computer-intensive models when you know that the computer can handle it, but for most applications the computer simulation speed is no longer always the over-riding concern any more.
In general, you'll have two opposing pressures:
Too few iterations and you get inaccurate outputs, graphs (particularly histogram plots) that look "scruffy';
Too many iterations and it takes a long time to simulate, and it may take even longer to plot graphs, export and analyze data, etc afterwards. Export the data into Excel and you may also come upon row limitations, and limitations on the number of points that can be plotted in a chart.
For general risk models, there is certainly a minimum number one should run. More technical arguments can be put into practice for determining how many iterations you need to run for a particular level of accuracy of some reported statistic, discussed below.
There will usually be one or more statistics that you are interested in from your model outputs, so it would be quite natural to wish to have sufficient iterations to ensure a certain level of accuracy. Typically, that accuracy can be described in the following way:
"I need the statistic Z to be accurate to within +/- d with confidence a."
These links describe how you can determine the number of iterations you need to run to get some specified level of accuracy for the most common statistics:
The examples contain models to let you monitor the level of accuracy with Crystal Ball in real time. The models make use of a few unusual functions:
CB.IterationsFN() | Crystal Ball function that returns the iteration currently being calculated during a simulation |
|---|---|
CB.GetForeStatFN(A1,5) | Crystal Ball function that returns the standard deviation of the values generated so far during the simulation in cell A1 |
CB.GetCertaintyFN(A1, x) | Crystal Ball function that returns the fraction of values generated so far during the simulation in cell A1 that are at or below x |
NORMSINV(x) | Excel function that returns a Z-value for any x-value on the Normal(0,1) (standard normal) distribution |
Note that all models assume that you are using Monte Carlo sampling. This will therefore somewhat over-estimate the number of iterations you'll need if you are using Latin Hypercube sampling (which we recommend, in general). That said, in practice Latin Hypercube sampling will only offer useful improvement when a model is linear, or when there are very few distributions in the model.
