*Variance = {(x_{1}-µ)^{2} + (x_{2}-µ)^{2} + ... + (x_{n}-µ)^{2}} / n; where µ is the Arithmetic Mean of the data set. |

*Variance = {(x_{1}-µ)^{2} + (x_{2}-µ)^{2} + ... + (x_{n}-µ)^{2}} / n; where µ is the arithmetic mean of the data set. |

:Editorial comment: it is actually the first formula that has precision problems when dealing with limited-precision arithmetic. If the difference between measurements and the mean is very small, then the first formula will yield precision problems, as information will be lost in the (x_{i} - µ) operation. There is no such loss of significance in the intermediate operations of the second formula. -- ScottMoonen: Editorial comment the second: in fact, the second formula is the one more commonly beset with problems. The first can have problems when the mean is very large relative to the variance, but this is relatively rare in practice and this problem also affects the second formula. Much more common is the situation where you have comparable mean and variance and a very large number of observations. In this case, the second formula will result in the subtraction of two very large numbers whose difference is relatively small (by a factor roughly equal to the number of observations). If you have one million observations, you lose roughly six significant figures with the second formula if you use ordinary floating point arithmetic. -- TedDunning:''comment: The problem may occur #when the deviations are very small relative to the mean or #when they are small relative to the representational capacity of the arithmetic instrument (floating point computer, fixed point calculator, paper and pencil). To be precise we have to specify the instrument and the nature of the data. -- DickBeldin'' |

back to Variance |

back to Variance /Talk? |