Thanks to Sam Buttrie for solving this problem.
I was using the following to accumulate randomization replicates:
>>>
# Put actual data statistic in first column of Stat.RandDist
assign("StatRandDist.mtx", ActualDataStat.vec, frame=0)
set.seed(1)
apply(1:Nrand,1, FUN=RandStat.f)
where
RandStat.f<- function(dummy) {
RandOrder<- sample(dim(y.mtx)[1])
{ process y[RandOrder,] to get RandStat.vec}
assign("StatRandDist.mtx", cbind(StatRandDist.mtx, RandStat.vec),
frame=0)
}
<<<
For large Nrand, I would ruin out of memory. Sam pointed out the
problem and suggected a solutions. I thought I was gaining something by
passing the function results through an object in frame 0. I don't
think this achieved anything and if I ever forgot to retrieve the saved
object before q()ing, it was gone. You only lose an object that was
constructed in an overnight job once. After this occured, I implemented
Sam's solution and it works.
Sam's message:
>>>>>>>>>
One thing that's probably hurting you is cbind-ing every time through
the
loop. It's much more efficient to allocate the memory for your results
once;
when you cbind every time, S-Plus has to allocate memory for one column;
then it abandons that and allocates a separate piece of memory for two
columns; then for three, and so on.
Try something like this:
sams.rand.stat.f <- function(thing) {
return (francois.process (y, order = sample (nrow(y.mtx)))) #the result
of francois.process is RandStat.vec above
}
Assume that the "rand stat" thing is a vector of length n.
sapply (1:100, sams.rand.stat) will produce a n x 100 matrix without the
cbinding and I'm pretty sure it will help.
<<<<<<<<<<
It does seem to help. Thanks.
-francois.
-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news
|