I've noticed that different S-PLUS sessions seem to have quite different
maximum amounts of memory accessible (S-PLUS 7.0 under Windows 2000).
E.g., in some sessions I can get about 1.9Gb, while in others I can get
about 1.3Gb. It always seems to be one of these two values, and I can
see little reason for the difference. I initially expected that I would
be able to get less memory when I had more programs running, but this
doesn't seem to be the case, as I see the following sequence:
Start up S-PLUS with working directory A
-- can get 1256Mb allocated
Start up another S-PLUS session with working directory B (while still
keeping the first session running)
-- can get 1859Mb allocated
The only consistency I can find is that the maximum memory allocation
seems to depend on the working directory (i.e., with working directory A
I can only get 1256Mb, whereas with working directory B I can usually
get 1859Mb). The number of objects in a working directory doesn't seem
to matter -- if I delete all objects in working directory A and then
close and restart S-PLUS, I can still only allocate 1256Mb. Whether or
not I have other S-PLUS databases attached doesn't matter either.
I'm wondering if anyone has any clues as to why S-PLUS sessions seem to
have different limits in the amount of memory that can be allocated?
I'd like to be able to reliable allocate the maximum (e.g., in the
region of 1859Mb).
This is running S-PLUS 7.0.6 GUI (Enterprise version) under Windows
2000, working with the command line window. I do not see this behavior
with S-PLUS 6.2 -- it seems to be able to reliably allocate over 1.8Gb
of memory. (And yes, I know about the bigdata objects.)
Here's a function that exhausts memory, and prints a summary. If
there's some better way or reporting the memory usage, please let me
know. (I would regard an S-PLUS session in which this has been run as
"used up" -- I would restart before doing any serious work, as the
S-PLUS memory allocation table may be somewhat fragmented.)
exhaust.mem <- function(from=128, to=1, units=2^20, verbose=F) {
# Exhaust memory by allocating as many vectors as possible,
# starting with vectors of (from*units) bytes, and halving
# the size of vectors when no more can be allocated. Stop
# when we get down to vectors of less than (to*units) bytes.
# Return a summary of the memory allocated.
mem.tally.reset()
# find a nice name for the units (which are in bytes)
i <- which(2^c(30,20,10,0) <= units)[1]
m <- units / 2^(10*(4-i))
units.desc <- paste(if (m!=1) format(round(m, digits=2)),
c("G","M","K","")[i], "b", sep="")
# how many different sized chunks? set up a vector to count them
n <- floor(log(from/to)/log(2)) + 1
count <- integer(n)
names(count) <- format(round(from / 2^seq(len=n, from=0),
digits=2), justify="none")
total <- 0
k <- 1 # indexes count
# a list to store all the memory chunks we allocate
chunks <- vector("list", 1000)
chunk.i <- 1
# start trying to allocate the biggest chunks first
size <- from
while (size >= to) {
while (!is(try(x <- double(size * units / 8)), "Error")) {
if (verbose)
cat("Allocated", size, "\n")
chunks[[chunk.i]] <- x
chunk.i <- chunk.i + 1
if (chunk.i > length(chunks))
chunks <- c(chunks, vector("list", 1000))
total <- total + size
count[k] <- count[k] + 1
}
# get ready for the next iteration
size <- size/2
k <- k+1
}
list(units=c(size=units.desc, bytes=units), count=count,
total=total, mem.tally=mem.tally.report()[2]/units)
}
Here's an example of the output:
exhaust.mem()
$units:
size bytes
"Mb" "1048576"
$count:
128 64 32 16 8 4 2 1
11 2 5 4 3 11 10 10
$total:
[1] 1858
$mem.tally:
evaluation
1858.044
This means that 11 blocks of 128Mb were allocated, 2 blocks of 64Mb,
etc. The total allocated was 1858Mb, and mem.tally.report() reported
that maximum memory usage was 1858.044Mb.
|