Thanks go to Madeline Bauer, John Kershaw, Steve McKinney, and Thomas
Jagger!! I ended up splurging and getting 4GB of RAM, the Pentium 4
3.4GHz processor, and and a 250 GB hard drive.
Based on the responses, 4GB is probably, but not necessarily, overkill if
I'm just running Splus on a Windows XP Pro desktop. If I'm understanding
things correctly, it's Microsoft that puts the 2GB memory limit on
applications, but there are ways around it (see below). So, it sounds
like more memory will let me do more multitasking. The new architecture
of SPLUS 7.0 will make it less likely that I'll need large amounts of
memory even for very large datasets. As far as dual processors go, it
sounds like Splus won't take advantage of that, but you can set it up so
that Splus gets a dedicated processor which will help a great deal.
There was no mention of the use of hyperthreading.
I've included the specific responses, and my original question, below.
Thanks for your help!
Cheers,
Andy
***********************
I have a HP xw4100 workstation with 4 GB of Kingston (recommended) running
Win XP Pro. There is currently some fluke in XP that limits the amount of
ram it "sees" to about 3.5 GB. I went ahead and bought the max 4 GB
assuming that it was easier than messing around later and that, sooner or
later, the operating systems will catch up with technology.
You didn't mention hard drives, but surely you'll want big and fast. I
recently added an external Western Digital 250 GB for handy backups.
**********************
I currently use Splus 6.2 on Windows XP. Both my home and work computers
have 4GB of RAM (400Mhz). I work with very large data sets (typically
70,000+ records with 20+ fields) with no real performance problems. My
home machince is 2.4GHz and at work I have a 3.2GHz. I also have a tablet
PC running Windows XP for tablet, it only has 2GB of RAM and a 1.8
Centrino (about 2.4GHz Pentium). I also run the same data sets on the
tablet (in fact the tablet is becoming my primary machine - I really like
the writing capability). Occasionally, the screen will blank out as Splus
crunches numbers - loops are killers, but I find performance to be
acceptable. I multitask all the time - word process, web, email etc. and
never have too many slow downs or non-response periods. The only real
bottlenecks I have had is running Splus and Adobe Illustrator
simulataneously and working with heavy graphics - they are both memory
hogs and my tablet does complain and occasionally becomes non-responsive.
Splus or R is so far superior to anything else, I cannot complain the few
times my computer slows down. 4GB should be pleanty and 2GB would
certainly be sufficient for 90% of your operations.
***********************
I have a laptop with dual processors, 2GB ram. I can see that I use about
1.5 GB of the ram for some tasks that do not use all the RAM. When Splus
hogs one cpu, I still have the other to perform routine activities, such
as e-mail and web browsing. I don't like to hear the sound of my hard
drive swapping memory from the virtual page file to RAM and back so if I
can get away without a page file things run faster.
Also check on the options("memory") and options("object.size"), these may
default to use all of your memory. If Splus exceeds this value it can
terminate. This is a good thing. I assume the maximum is 4GB, (2^32) mine
is set to 2GB by default as this is my maximum size. If you have a swap
file Splus will use virtual memory. See chapter 21 in the Splus 6.0
Window's Programmer's Guide. (As a quick reference on memory usage).
Now, this may just be my problem, but with 1GB of RAM I can hibernate
fine, but with 2GB ram, my computer fails to hibernate. This has been
reported. It may be fixed I am running a Dell Inspiron 9100 with 2GB ram,
3.12 GHz, dual processor Intel 4 bought last July.
**********
The Windows operating system is 32 bit, so can address up to 4 billion
(2^32) memory locations. Half of this address space is reserved for
Windows, for doing graphical and other OS activities. Thus a user
application only has 2GB of addressable space. So unless you plan on
doing concurrent memory-intensive tasks, 2GB of RAM will be plenty, 4GB
will rarely if ever all be used.
The big data stuff does not require huge amounts of RAM, as it is all
designed to work on 'chunks' of data rather than everything all at once.
S-PLUS big data calls will use less memory than S-PLUS standard calls, as
the latter are in-memory and require all data to be loaded.
Note that S-PLUS itself is not limited to 2GB. 32 bit versions of S-PLUS
can handle up to 4GB, the upper limit imposed by 32 bit addressing. (Also
note that big data methods get around this limit by using 'incremental
update' algorithms on successive portions of the data, each of which is
small enough to allow all computation to happen with less than 4 GB of
addressable space.)
Many Unix/Linux operating systems allow user applications to address 3GB
of virtual memory space without doing anything special.
Microsoft has a couple of mechanisms to allow user applications to have
access to more than 2GB of addressable space. One is PAE (Physical
Address Extension) and the other is the /3GB switch (which enables user
applications to have access to 3GB of flat virtual address space). Both
are discussed in the webpage corresponding to this link:
http://www.microsoft.com/whdc/system/platform/server/PAE/PAEmem.mspx
By setting a /3GB switch in the boot.ini file, programs which have had the
bit IMAGE_FILE_LARGE_ADDRESS_AWARE set in their image header can access
3GB of memory. S-PLUS 7.0 has this bit set, so is capable of addressing
3GB of memory. Thus on server machines it is possible to access an
additional GB of memory. This is discussed in the S-PLUS 7.0 release
notes in the section "Support for 4GT RAM Tuning".
http://www.insightful.com/support/splus70win/ReleaseNotes.htm
Microsoft claims the /3GB switch is available on Windows XP Professional,
though developers at Insightful have had problems running S-PLUS 7.0 on
Windows XP Professional with the /3GB switch enabled, so Insightful does
not support that configuration.
So if you have access to S-PLUS on one of the server operating systems
listed below, you could set the 3GB switch and gain access to an
additional GB of addressable space.
Now back to your original physical RAM question: You can use the /3GB
switch even if you only have, say, 512 MB of physical RAM. You'll need to
configure your Windows system swap file (pagefile.sys) to be able to
handle the rest, so it would need to be 3GB or larger. As long as your
physical RAM plus swap file capacity is large enough, S-PLUS will be able
to address up to 3GB of virtual memory space. Since disk I/O is slow,
it is best to have as much physical RAM as possible.
That said, it is rare that you would be operating on one object larger
than about 1 GB in S-PLUS. Any object larger than that will be handled in
"chunks" with the new big data objects. Any object larger than that
handled by older S-PLUS methods, which are all "in-memory", will require
that copies be made, so you'll just run out of memory. Hence in-memory
methods can only operate on objects up to a few hundred megabytes, so at
any given time S-PLUS will only need a few hundred megabytes to do its
in-memory work efficiently, without doing a lot of disk I/O. This is why
you can get away with working on fairly large objects with only 760 MB or
1 GB of RAM. Thus I did not suggest that you put in 3GB of RAM, as you
typically won't notice much of a performance enhancement beyond 1 GB or 2
GB of RAM unless you are often working on datasets of tens to hundreds
of megabytes.
The new big data methods, because they use 'incremental update'
algorithms, do not require that all data be in-memory at once. Thus the
big data methods typically use only a few hundred megabytes of RAM, even
when working on gigabyte-sized datasets. So you won't need 3 or 4 GB of
RAM to effectively run the new big data methods.
Only if you will need to regularly process datasets of tens to hundreds of
megabytes in size with in-memory methods will you benefit from having 3 or
4 GB of RAM.
(In my opinion, if the additional RAM is cheap, throw it in. If it is
costly, save the money for other purposes.)
************************
Hi All,
This question has been addressed many times, but not recently and definitely
not in regard to S-PLUS 7. I am in the fortunate position of getting a new
desktop (unfortunately it'll have to run Windows XP).
Essentailly, my question is, is there such thing as too much RAM, especially
given S-PLUS 7's "pipeline architecture" and "big data library" (which I know
next to nothing about)? Is 2GB enough? Is 4GB overkill? Whatever amount
of memory I get, I'm thinking of 533MHz SDRAM rather than the 400 MHz.
Right now, I'm looking at a Pentium 4 Processor, 3.4 GHz, 1MB cache, and 800
MHz FSB. Previous messages in the Archives suggest that dual processors or
hyperthreading doesn't help much with SPLUS (unless you're having the
computer perform multiple tasks such as web browsing or word processing while
running SPLUS). Is that still the case? I'm guessing given other messages
in the Arhives that going to 3.6 GHz processor probably wouldn't be
noticable.
Are there other bottlenecks I should be aware of? Any advice would be very
helpful!
Thanks,
Andy
******************************************************************
"What if the Hokey Pokey is all it really is about?" - Jimmy Buffett
Andrew B. Cooper, Ph.D.
Department of Natural Resources
Nesmith Hall, Rm 208A
University of New Hampshire
Durham, NH 03824
andrew.cooper@unh.edu
603.862.4254, 603.862.4976 (FAX)
http://www.unh.edu/natural-resources/fac-cooper.html
******************************************************************
--------------------------------------------------------------------
This message was distributed by s-news@lists.biostat.wustl.edu. To
unsubscribe send e-mail to s-news-request@lists.biostat.wustl.edu with
the BODY of the message: unsubscribe s-news
|