Has anyone found a trick to importing really large txt files into S+ 6.2
under XP? I sent the question to Insightful and their only recommendation
was to break it up into smaller files. The file is 350 megs, which is large
I grant, but my machine has 4 gigs of memory. If I did want to break it up,
what utility could I use to do so? Excel is not going to read it either.
See below for my full question and support's answer. Thanks in advance.
I get the "unable to obtain requested dynamic memory" error when I try to
read in a large file into S+ 6.2 using the following command:
data <-
read.table("M:\\tina\\R2000V10SPLS29m.TXT",header=TRUE,sep=",",as.is=TRUE,na
strings="NA")
dim(data)
The text file is 347,456 KB big. My windows XP machine has 4 Gigs of
memory, which I believe is the max it can handle. I also believe that my
virtual memory is maxed out. I read the FAQ on this topic, but it mostly
said to optimize the code and I am just trying to read it in. I understand
that the operating system steals half of this. Do I need to change any
setting to make sure S+ is fully utilizing my memory capabilities? Anything
else I can try?
----------------------------------------------------------------------------
-----------
Solution:
The file you are trying to import is a very large file. The calculation we
use to calculate the size of the data you are trying to import is:
(rows)*(columns)*8*4.5
You should import the file by breaking it into smaller files. Then import
these smaller files into S-Plus and finally, recombine them inside S-Plus.
Please let me know if you have any questions.
Sincerely,
Jacob Geballe
===========================================================================
Jacob Geballe email: support@insightful.com
Technical Support Engineer FAX: (206) 283-8691
Insightful Corporation Phone: (206) 283-8802 ext.235
www.insightful.com 1-800-569-0123 ext.235
===========================================================================
Roger J. Bos, CFA
Rothschild Asset Management
1251 Avenue of the Americas
New York, NY 10020
212-403-5471
********************************************************************** * This
message is for the named person's use only. It may
contain confidential, proprietary or legally privileged
information. No right to confidential or privileged treatment
of this message is waived or lost by any error in
transmission. If you have received this message in error,
please immediately notify the sender by e-mail,
delete the message and all copies from your system and destroy
any hard copies. You must not, directly or indirectly, use,
disclose, distribute, print or copy any part of this message
if you are not the intended recipient.
**********************************************************************
|