I don't see a way out of a for loop, but you can do it much more
efficiently. "lm" does a lot of things that you only really need to do
once. An outline of an alternative computation is:
# create big.xmat (including intercept)
for(i in 1:nsub) {
# create subscript for i, call it this.sub
this.lm <- lm.fit.qr(big.xmat[this.sub, , drop=F], y[this.sub])
slope[i,] <- coef(this.lm)[2]
}
Page 61 of S Poetry has a bit on this.
Patrick Burns
Burns Statistics
patrick@burns-stat.com
+44 (0)20 8525 0696
http://www.burns-stat.com
(home of S Poetry and "A Guide for the Unwilling S User")
Pravin wrote:
Sorry for the errors in my previous email. The loop should read--
subset=sub==i (line 3)
slope[i,]<-coe(od.fit)[2]} (line4)
-----Original Message-----
From: s-news-owner@lists.biostat.wustl.edu
[mailto:s-news-owner@lists.biostat.wustl.edu] On Behalf Of Pravin
Sent: Monday, March 01, 2004 7:39 PM
To: s-news@wubios.wustl.edu
Subject: [S] S-PLUS Vs some other softwares
Hi all,
(Almost)Always I have written S-PLUS code where for() loop looked
indispensable to ME. Since it did my job at the expense of
slightly more dos time, I never looked at the alternatives. But,
this time I have a very simple problem and I thought for() loop
should be able to do the job. But it didn't!
I am doing one permutation experiment that requires me to analyze
data from 500,000 patients(9 samples per patient) and all I want
to fit is the linear regression model and extract the estimates of
slope on each patient. After running my computer for 16 hrs (CPU
usage looked like it was computing all the time), S-PLUS reached
patient number 19,000&...Is there any quicker way of doing this in
S-PLUS? Or from what I always hear---- S-PLUS is limited by its
ability to handle huge datasets at hand, do I have to look for
some other software that can do this huge computational task
really quickly? Any recommendations?
LOOP:
Nsub<-500,000
for (i in 1:nsub)
{ od.fit<-lm(data.y~data.x,data=data.ram,subset=sub,singular.ok=T)
slope[i,]<-coe(od.fit)[2}}
Thanks much,
Pravin
Pravin Jadhav
|