s-news
[Top] [All Lists]

Re: Internal rounding errors

To: courses@zanybooks.com
Subject: Re: Internal rounding errors
From: Thomas Lumley <tlumley@u.washington.edu>
Date: Tue, 3 Mar 2009 07:45:37 -0800 (PST)
Cc: s-news@lists.biostat.wustl.edu
In-reply-to: <20090303083828.b21645f2bceb2bcc4cba8e709837b1d9.8d29d45cf1.wbe@email.secureserver.net>
On Tue, 3 Mar 2009 courses@zanybooks.com wrote:

Using R (not S-Plus) yields the following:
> data3=1:25+rnorm(25)
> temp=data3-mean(data3)
> sum(temp)
[1] 1.154632e-14


And so?

sum(1:25)*.Machine$double.eps
[1] 7.21645e-14

so the rounding error is about six times smaller than a bound based on machine 
precision, and is very close to mean(1:25)*sqrt(25)*.Machine$double.eps, which 
seems a reasonable expected value for the internal rounding error.

You weren't expecting zero, were you?

    -thomas

Thomas Lumley                   Assoc. Professor, Biostatistics
tlumley@u.washington.edu        University of Washington, Seattle



<Prev in Thread] Current Thread [Next in Thread>