s-news
[Top] [All Lists]

Re: Calculation glitch?

To: "'Steve Su'" <s.su@qut.edu.au>, s-news@lists.biostat.wustl.edu
Subject: Re: Calculation glitch?
From: "Austin, Matt" <maustin@amgen.com>
Date: Fri, 27 Aug 2004 18:55:58 -0700
Steve,
 
You are battling something you can't win.  The threshold value which can cause undetected overflow errors is > 2^(b-1)-1 where b is 32 for 32-bit systems.  The largest value you can reliably represent as an integer is 2,147,483,647.  You can verify this with by looking at
 
> .Machine$integer.max
[1] 2147483647
 
The number of 2,147,483,648 could (would?) overflow and could go back to -1.
 
A decent intro reference for these kind of issues is Numerical Issues in Statistical Computing for the Social Scientist by Altman, Gill and McDonald.
 
--Matt

-----Original Message-----
From: s-news-owner@lists.biostat.wustl.edu [mailto:s-news-owner@lists.biostat.wustl.edu]On Behalf Of Steve Su
Sent: Friday, August 27, 2004 18:30 PM
To: s-news@lists.biostat.wustl.edu
Subject: [S] Calculation glitch?

Dear All,
 
I am using SPLUS Professional Edition Version 6.0.3 Release 2 for Microsoft Windows XP.
 
I found that if I do:
 

>1000000*1500000

# I get:

[1] NA

 

A workaround is to do:

 

>1000000.0*1500000.0

[1] 1.5e+012

 

Is there a way I can correct for the above bug without putting decimal places?

 

Thank you for your attention.

 

PS. No luck in getting SPLUS 6.1 in my university yet, as they are out of licence!

 

Steve.
<Prev in Thread] Current Thread [Next in Thread>