s-news
[Top] [All Lists]

Summary: Memory Accumulation

To: "S-PLUS Newsgroup" <s-news@lists.biostat.wustl.edu>
Subject: Summary: Memory Accumulation
From: "Crawford.Winnie" <crawford.winnie@ensco.com>
Date: Wed, 25 Feb 2009 17:16:21 -0500
Thread-index: AcmXlq+bfxNv3j97SV2PROCI0qMIZA==
Thread-topic: Summary: Memory Accumulation

All,

 

This is a summary of the responses to my email, and a summary of my findings after trying one of the suggestions. As a reminder, I am running S-Plus 8 on a Windows XP PC with 4 GB RAM.

 

Thanks to the responders Glenn Treacy, Chuck Taylor, Kim Elmore, Chris “from Taipei”, Alan Hochberg, Robin Reed, and Stephen Kalzuny of Tibco. First, I learned that the proper term is ‘leak’ and not ‘accumulation’. Most responses included “I feel your pain” statements. I know this has been a well-known S-Plus issue for a long time. I’ve managed to dance around it by running my scripts piece-meal, but I have reached the piece-meal limit.

 

One suggestion received said to read the chapter “Using Less Time and Memory” in the Programmer’s Guide. The exact process of how S-Plus allocates memory was over my head, and I was already familiar with most of the suggestions on how to streamline code. It’s a good reference for those who have not read it before. Another suggestion advised me to not use rbind() or cbind() – my script does us rbind() – but to build a dataframe and make assignments to it. The problem is that I do not know how large the resulting dataframe will be. I haven’t tried it yet, but may play with it later. Another suggestion was to set up a batch run by going to Program Files > S-Plus 8.0 > S-Plus Batch (or TIBCO > Spotfire S + 8.1 > Spotfire S + BATCH). I could not find this in my directory structure, so I haven’t tried it yet.

 

The suggestion I did try came from two responders, Kim and Robin, to run the script in the Commands window. I did a test and found that there was still some memory leakage, but about 50-75% less than when running the script. Then I started deleting data frames I no longer needed. The leaked memory was 13,347,920 before I started and 196,719,856 when I finished deleting files. So, there is a memory leak even when deleting frames. If anyone wants to do a little experiment and see how much memory is leaking from simple commands to complicated scripts, run memory.size() at the very beginning of your session to see your base memory, then run it every so often throughout your session.

 

As I stated in the original email, the S-Plus help recommends quitting and re-invoking S-Plus if ‘things get out of hand.’ This is a harsh solution and, in my opinion, not appropriate for a sophisticated and expensive software package. I hope that the new owners will take a look at this and fix the leak that has plagued S-Plusers for many years.

 

Thank you for listening and helping.

 

Win

 

*****************************************************************

Winifred C. Crawford  Staff Scientist/Senior Meteorologist

ENSCO, Inc.

Aerospace Sciences and Engineering Division

1980 N. Atlantic Ave., Suite 830

Cocoa Beach, FL  32931

VOICE:  321.853.8130  FAX:  321.853.8415

EMAIL:  crawford.winnie@ensco.com

 

AMU Quarterly Reports are available online:

http://science.ksc.nasa.gov/amu

*****************************************************************

 

 
The information contained in this email message is intended only for the use of the individual(s) to whom it is addressed and may contain information that is privileged and sensitive. If you are not the intended recipient, or otherwise have received this communication in error, please notify the sender immediately by email at the above referenced address and note that any further dissemination, distribution or copying of this communication is strictly prohibited.

The U.S. Export Control Laws regulate the export and re-export of technology originating in the United States. This includes the electronic transmission of information and software to foreign countries and to certain foreign nationals. Recipient agrees to abide by these laws and their regulations -- including the U.S. Department of Commerce Export Administration Regulations and the U.S. Department of State International Traffic in Arms Regulations -- and not to transfer, by electronic transmission or otherwise, any content derived from this email to either a foreign national or a foreign destination in violation of such laws.
<Prev in Thread] Current Thread [Next in Thread>
  • Summary: Memory Accumulation, Crawford.Winnie <=