The Bootstrap (sometimes called Resampling) was introduced by Efron (1979) and is explored in great depth and very readably in both Efron and Tibshirani (1993) and Davison and Hinkley (1997). It is an extremely flexible technique belonging to the classical school of statistics.
This section presents a brief introduction covering the main aspects of the bootstrap. Key reasons for the popularity and usefulness of the bootstrap are:
It is easy to perform using Monte Carlo simulation methods;
It corresponds well with traditional techniques where they are available, particularly when a large data set has been obtained; and
It offers an opportunity to assess the uncertainty about a parameter (especially when multiple parameters are correlated) where more traditional classical statistics techniques are not available.
The precursor to the Bootstrap
The non-parametric Bootstrap is used to estimate parameters of a population or probability distribution when we do not know the distributional form, which is the most common situation.
The parametric Bootstrap is used to estimate parameters of a population or probability distribution when we believe we know the distributional form (e.g. Normal, Lognormal, Gamma, Poisson, etc).
Using the Bootstrap as a likelihood function in Bayesian inference
We are sometimes interested in estimating parameters that describe relationships between variables, for example: regression parameters and rank correlation coefficients. The Bootstrap can provide uncertainty about these estimates in an intuitive way, by Bootstrapping the paired data values.