SUMMARY: S-PLUS Vs some other softwares

["Pravin" <jadhavpr@vcu.edu>]

Thank you very much for all the comments!

 

Fortunately/Unfortunately there was no suggestion to switch to some other software. Finally,,,, S-PLUS did the job.

 

Compared to the use of for() loop---see below---these functions help to surpass annoyingly lengthy loops.

 

Solution 1: results<- by(data.ram,data.ram$sub,function(ppr)(lsfit(ppr$x.var,ppr$y.var)$coef)[2] )

        Time = 19 mins, Job done!

 

Solution 2: use of "lmList"

Might work but it looked like I am limited by hardware configuration. I was surprised to see memory requirements for S-PLUS.

Computer ran into error- "Unable to obtain requested dynamic memory"

 

Here are my computer details: HP Notebook PC Intel(R), Pentium(R) 4-- CPU 2.00 GHZ, 256MB RAM and OS-WinXP

But I am certain that it would work for some other case, this is what I gathered after reading its documentation.

 

Keep in mind, this dynamic memory includes paged memory as well(over and above RAM) because there was windows error message about limited memory in addition to the mentioned SPLUS error.

 

Solution 3: use of "lm.fit.qr"

Again documentation explains possible application in such cases. I didn't implement because it was not easy to write i subscript due to variable number of data points/patient (varying between 5 to 9). But can find its application in some other case.

Pravin

Pravin Jadhav

-----Original Message-----
From: Pravin [mailto:jadhavpr@vcu.edu]
Sent: Monday, March 01, 2004 7:47 PM
To: 's-news@wubios.wustl.edu'
Subject: RE: [S] S-PLUS Vs some other softwares

Sorry for the errors in my previous email. The loop should read--

subset=sub==i (line 3) 

slope[i,]<-coe(od.fit)[2]} (line4)

 -----Original Message-----
From: s-news-owner@lists.biostat.wustl.edu [mailto:s-news-owner@lists.biostat.wustl.edu] On Behalf Of Pravin
Sent: Monday, March 01, 2004 7:39 PM
To: s-news@wubios.wustl.edu
Subject: [S] S-PLUS Vs some other softwares

Hi all,

(Almost)Always I have written S-PLUS code where for() loop looked indispensable to ME. Since it did my job at the expense of slightly more dos time, I never looked at the alternatives. But, this time I have a very simple problem and I thought for() loop should be able to do the job. But it didn't!

I am doing one permutation experiment that requires me to analyze data from 500,000 patients(9 samples per patient) and all I want to fit is the linear regression model and extract the estimates of slope on each patient. After running my computer for 16 hrs (CPU usage looked like it was computing all the time), S-PLUS reached patient number 19,000…...Is there any quicker way of doing this in S-PLUS? Or from what I always hear---- S-PLUS is limited by its ability to handle huge datasets at hand, do I have to look for some other software that can do this huge computational task really quickly? Any recommendations?

LOOP:
Nsub<-500,000
for (i in 1:nsub)  
        { od.fit<-lm(data.y~data.x,data="">
        slope[i,]<-coe(od.fit)[2}}
               
Thanks much,

Pravin

Pravin Jadhav