Hi,
I use splus pro 7.0 "professional developer" under windows XP (dell optiplex /
3.2GHz pentium 4 / 1.0GB RAM)
Lately I'm encountering an increasing number of projects that involve large
data sets, e.g. data frames with >million records and dozens of fields. I'm
running into problems characterized by this error message:
Problem in .A0[ri] <- .A1: Unable to obtain requested dynamic memory
This error occurs in various situations, most recently with a simple call to
rbind.data.frame() to append several hefty data frames.
I realize that I may be able to avoid some memory-related problems by using
better programming techniques. For now, let's assume that is not a practical
option. I've investigated tuning some parameters that control paging on my PC,
and I've tweaked everything towards "bigger" as best I can manage.
I am considering addressing the problem by upgrading to Insightful's
"enterprise edition" in order to have access to the "big data" features of that
edition. Before I spend big money to acquire big data capabilities, I'd like to
hear any comments or recommendations from users who are familiar with the
enterprise edition.
What are your impressions of the enterprise edition, especially the big data
features? Is that appropriate software for solving the kind of problem
described above? If so, what kinds of new problems could arise by migrating to
that package? How difficult would it be to adapt existing code scripts to
exploit the big data routines? Does using enterprise edition introduce any
significant new requirements for writing decent scripts? In general, what
pitfalls might be lurking in enterprise edition that could come as an unwelcome
surprise?
Are there any obvious or not-so-obvious tricks that I could use to minimize
memory problems without going to enterprise edition? Are there any new features
of splus pro v8 that might help with these problems?
Thanks,
dc
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dave Cacela 303-381-8000 (voice)
Managing Scientist 303-381-8200 (fax)
Stratus Consulting
PO Box 4059
1881 Ninth St
Boulder, Colorado 80302
mailto://dcacela@stratusconsulting.com
http://www.stratusconsulting.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|