---
title: "Linear regression non-parametric Bootstrap"
canonical: "https://modelassist.epixanalytics.com/space/EA/26575416/Linear%20regression%20non-parametric%20Bootstrap"
format: markdown
---
A slight modification to the [linear regression parametric Bootstrap](https://epixanalytics.atlassian.net/wiki/spaces/EA/pages/26575417) allows one to use a non-parametric Bootstrap, i.e. where we can remove the assumption of Normally distributed residuals which may often not be very accurate. For the non-parametric model, we must first develop a non-parametric distribution of residuals by changing them to have constant variance. We define the *modified residual* *r*<sub>*i*</sub> as follows:

 

![image](media://8ea44861-8d38-42e5-8377-e743e8e58401)

> Macro (mathblock)

 

<span style="color: #000000">where the </span><span style="color: #000000">*leverage*</span><span style="color: #000000"> </span><span style="color: #000000">*h*</span><sub><span style="color: #000000">*i*</span></sub><span style="color: #000000"> is given by:</span>

 

![image](media://7e05abba-207f-4914-8d19-ab010f22478a)

> Macro (mathblock)

 

The mean of the modified residuals > Macro (mathinline)

  is calculated. Then a Bootstrap sample *r*<sub>*i*</sub>* is drawn from the set of *r*<sub>*i*</sub> values and used to determine the quantity  > Macro (mathinline)



![inlinewrhat.JPG](media://c3000bd7-7f5a-4bdc-8707-1847d1ccb67a)

 for each *x*<sub>*j*</sub> value which is used in step 2 of the algorithm above. The model [Non-Parametric Regression Bootstrap](#Non-parametric_regression_Bootstrap) provides an illustration.

> Macro (anchor)



The links to the Non-Parametric Regression Bootstrap software specific models are provided here:

<details>
<summary>Crystal Ball</summary>

> Macro (inline-media-image)

 [Non-parametric_regression_Bootstrap](/wiki/download/attachments/26579343/Non-parametric_regression_Bootstrap-CB.xlsx?version=1&modificationDate=1506113498000&cacheVersion=1&api=v2)
</details>

<details>
<summary>@Risk</summary>

> Macro (inline-media-image)

 [Non-parametric_regression_Bootstrap](/wiki/download/attachments/26581760/Non-parametric_regression_Bootstrap-AtRISK.xlsx?version=1&modificationDate=1506097518000&cacheVersion=1&api=v2)
</details>

 

<span style="color: #000000">In certain problems, it is logical that the </span><span style="color: #000000">*y*</span><span style="color: #000000">-intercept value </span><span style="color: #000000">*c*</span><span style="color: #000000"> be set to zero. In this situation, the leverage values are different:</span>

 

![image](media://1a68d71d-0d85-4c7b-853c-ab4f2a1a5d7b)

> Macro (mathblock)

 

<span style="color: #000000">The modified residuals are thus also different and won't sum to zero, so it is essential to mean-correct the residuals before they are used to simulate random errors.</span>

 

Bootstrapping the data pairs is more robust than Bootstrapping the residuals as it is less sensitive to any deviation from the regression assumptions, but won't be as accurate where the assumptions are correct. However, as the data set increases in size, the results from Bootstrapping the pairs approaches that for Bootstrapping the residual and it is also easier to execute, of course. These techniques can be extended to non-linear, non-constant variance and to multiple linear regressions, described in detail in [Efron and Tibshirani (1993)](https://epixanalytics.atlassian.net/wiki/spaces/EA/pages/26579303) and [Davison and Hinkley (1997)](https://epixanalytics.atlassian.net/wiki/spaces/EA/pages/26579303).

 

 

 

---