The Bootstrap
The Bootstrap (sometimes called Resampling) was introduced by Efron (1979) and is explored in great depth and very readably in both Efron and Tibshirani (1993) and Davison and Hinkley (1997). It is an extremely flexible technique belonging to the classical school of statistics.
This section presents a brief introduction covering the main aspects of the bootstrap. Key reasons for the popularity and usefulness of the bootstrap are:
It is easy to perform using Monte Carlo simulation methods;
It corresponds well with traditional techniques where they are available, particularly when a large data set has been obtained; and
It offers an opportunity to assess the uncertainty about a parameter (especially when multiple parameters are correlated) where more traditional classical statistics techniques are not available.
The precursor to the Bootstrap
The non-parametric Bootstrap
The non-parametric Bootstrap is used to estimate parameters of a population or probability distribution when we do not know the distributional form, which is the most common situation.
Example:
Estimate of population mean, standard deviation and other statistics
The parametric Bootstrap
The parametric Bootstrap is used to estimate parameters of a population or probability distribution when we believe we know the distributional form (e.g. Normal, Lognormal, Gamma, Poisson, etc).
Examples:
Estimate of population mean, standard deviation and other statistics
Using the Bootstrap as a likelihood function in Bayesian inference
Estimating parameters for multiple variables
We are sometimes interested in estimating parameters that describe relationships between variables, for example: regression parameters and rank correlation coefficients. The Bootstrap can provide uncertainty about these estimates in an intuitive way, by Bootstrapping the paired data values.
Examples:
Estimate of least squares regression parameters
Difference between two population means
