At 12/12/2001 at 05:34 PM, Sumithra Mandrekar wrote:
Hi,
I am encountering a strange problem while using the
following S-Plus routine,
For some reason, when it compares 12.9, 13.3, and 13.7
which is the (2, 3, 5) triple comparison, the mean is
13.3 and median is 13.3 and so x should be 0, instead
the value of x is calculated as: 1.776357e-015.
Can someone please tell me why this is happening?
What is happening is that you are doing floating-point math and your result
is essentially zero. If you need to check for equality of floating point
quantities, it is best to do it within some tolerance, e.g. (more in
fortran than s):
tol = 1.0e-10
if ((x1 - x2) / (x1 + x2) ) < tol then (do something)
See also
http://www.lahey.com/float.htm
--
Michael Prager, Ph.D. <Mike.Prager@noaa.gov>
NOAA Center for Coastal Fisheries and Habitat Research
Beaufort, North Carolina 28516
http://shrimp.ccfhrb.noaa.gov/~mprager/
|