To learn more about EpiX Analytics' work, please visit our modeling applications, white papers, and training schedule.
Page tree
Skip to end of metadata
Go to start of metadata

 

 

Assume that we have a set of n data samples from a Normal distribution with unknown mean m and known standard deviation s. We would like to estimate the mean together with the appropriate level of uncertainty. A Normal distribution can have a mean anywhere in [-∞, +∞], so we could use a Uniform improper prior p(m) = k. This is a more unusual case than where one does not know the standard deviation but might occur, for example, if one was making many measurements of the same parameter, but believed that the measurements had independent, normally distributed errors and no bias (so the distribution of possible values would be centered about the true value).

 

We assign an uniformed (Uniform) prior for m and use a Normal likelihood function for the observed n measurements {xi}. No prior is needed for s since it is known and we arrive at a posterior distribution for m given by:

 

f(\mu)=\frac{1}{(2\pi \sigma ^{2})^{n/2}}exp\Big(-\frac{1}{2\sigma ^{2}}\displaystyle\sum_{i-1}^{n}(x_{i}-\mu )^{2}\Big)

 

Taking logs:

 

 

L(\mu )=log_{e}[f(\mu )]=-\frac{n}{2}log_{e}[2\pi ]-nlog_{e}[\sigma ]-\frac{1}{2\sigma ^{2}} \displaystyle\sum_{i-1}^{n}(x_{i}-\mu )^{2}


 

Since s is known:

 

L(\mu )=k-\frac{1}{2\sigma ^{2}}\displaystyle\sum_{i-1}^{n}(x_{i}-\mu )^{2}


 

where k is some constant. Differentiating twice we get:

 


\frac{dL(\mu )}{d\mu }=\frac{1}{\sigma ^{2}}\Bigg(\displaystyle\sum_{i-1}^{n}x_{i}-\mu n\Bigg)


 


\frac{d^{2}L(\mu)}{d\mu ^{2}}=\frac{-n}{\sigma ^{2}}


The best estimate m0 of m is that value for which :\frac{dL(\mu)}{d\mu }=0

 


 

\frac{dL(\mu)}{d\mu}\Big|_{\mu _{0}}=\frac{1}{\sigma ^{2}}\Big(\displaystyle\sum_{i-1}^{n}x_{i}-\mu_{0}n\Big)=0

 

i.e., m0 is the average of the data values x - no surprise there! A Taylor series expansion of this function about m0 gives:

 

     

L(\mu)=L(\mu_{0})+\frac{d^{2}L(\mu)}{d\mu ^{2}}\Big|_{\mu _0}\frac{(\mu -\mu _{0})^{2}}{2}=-\frac{n}{\sigma ^{2}}\frac{(\mu -\mu_{0})^{2}}{2}

     

                                                                                                 (1)

 

The second term is missing because it equals zero and there are no other higher order terms since \frac{d^{2}L{\mu}}{d\mu ^{2}}=\frac{-n}{\sigma ^{2}} is independent of m and any further differential therefore equals zero.

 

Consequently, Equation 1 is an exact result.

 

Taking exponents to convert back to f(m), and rearranging a little, we get:

 


f(\mu)=Kexp\Bigg(\frac{(\mu -\bar{x})^{2}}{2\Big(\frac{\sigma }{\sqrt{2}}\Big)^{2}}\Bigg)


 

where K is a normalizing constant. Comparison with the probability density function for the Normal distribution, shows that this is a Normal density function with mean x and standard deviation\sigma \Big/\sqrt{n}. In other words:

 


\mu= Normal\Big(\bar{x},\sigma \Big/\sqrt{n}\Big)


 

which is the same as the classical statistics result.

 

 

 


  • No labels