I have been having performance problems involving my simulation
applications on Splus 6.0, and was hoping some of you might be able
offer suggestions on how to speed things up. I've been developing and
using these applications for years with SPlus 3.4, and recently
switched over to Splus 6.0 on faster hardware. Now that much of my
application has been ported (which other than the [.data.frame issue
was pretty straighforward), it works correctly, but unfortunately it
is significantly slower.
I have abstracted the portions of the simulation that slowed down the
most into a short benchmark, below. For rowcount=10000 it takes about
15 seconds in Splus 3.4, 8 seconds in R, and 295 seconds in Splus 6.0.
For an earlier version of this benchmark, Insightful support had
suggested that there might be a performance issue involving how
reference counting is done in the new S engine (see below), but I
haven't heard anything beyond that.
I don't mean to suggest that Splus 6.0 is slower than Splus 3.4 in
general; it simply is the case that the mix of capabilities used by my
simulation applications happen to have slowed down. So for obvious
reasons, I'm focusing on describing that problem area in this message,
in hopes of finding a solution. I certainly have seen vectorized
calculations speed up in Splus 6.0 relative to 3.4. In some cases the
speed up is only available at larger vector sizes (which is generally
where you would think performance matters most), with small vector
sizes unable to fully amortize the increased overhead. So vectorized
operations on small vectors can run slower in Splus 6.0. This can be
a problem if the small vector manipulation is inside the inner loop of
a larger non-vectorizable simulation...
----------------
DETAILS:
My application is basically a loop doing a financial simulation, where
the calculation of iteration n+1 can depend on possibly everything
that has happened before that point in the simulation, or perhaps only
a recent window of several iterations of history. So I do not think
the loop can be vectorized. Other application characteristics that
might be important: Splus 3.4 encouraged you when tuning your code to
move the body of the simulation loop to a separate function leaving
behind as little as possible, which could improve garbage collection
as the inner function would reclaim some space each time it exited.
Also, the enclosing data structure is a often list whose elements are
dataframes, or occasionally lists of dataframes. The number of rows
is large (number of time periods), but the number of columns is fixed
(data needed at each iteration). In the benchmark below, the elements
are simply vectors.
Many of these characteristics would likely be common to many types of
simulation code, not only financial industry investment strategy
simulations.
Benchmark function:
benchmark.1 <- function(rowcount=5)
{
#load in/initialize the simulation data. In the general case, each
# element of the list might be a dataframe, i.e. with securities as
# rows and security characteristics as columns. The number of
# characteristics is generally fixed, but the number of securities
# can vary and be large, i.e. thousands. However, in this benchmark,
# each elemetn is simply a vector of 30 random numbers.
data <- lapply(1:rowcount,function(x){rnorm(30)})
#At end, this vector the results of each iteration.
#There can be thousands or more iterations, depending on the
#granularity of the time period. And in the general case,
#The result might resemble the input in structure, i.e. a list
#whose elements are dataframes, each processed version of the corresponding
#input dataframe and its predecessors.
results.at.each.iter <- rep(NA,rowcount)
for(i in 1:rowcount) {
#could be this
# results.at.each.iter <- benchmark.2(i,results.at.each.iter,data)
# But lets say for simplicity that it is this simpler call
# which just passes a single1 time period, or at worst a window of several
recent
# time periods, but not all of them:
results.at.each.iter[i] <-
benchmark.2(results.at.each.iter[max(i-1,1)],data[[i]])
}
results.at.each.iter
}
benchmark.2 <- function(prev.result,data)
{
#any arbitrary calculation might be here, so don't
#take advantage of the simplicity of this benchmark
#would likely return a vector with length equal to the number of
#securities, but here I just return one number.
if(is.na(prev.result) | (prev.result>4) | (prev.result<20)) {
new.result <- sum(data,na.rm=T)+ ifelse(is.na(prev.result),1,prev.result)
} else {
new.result <- sum(data)*prev.result
}
new.result
}
----------------
All timings below are in seconds. Splus 3.4 is on a slower machine,
an Ultra60 360 mhz/2 CPU/512K RAM. Both R and SPlus 6.0 are on an
E250 400 mhz/2CPU/2 Gig RAM). Timings done by
unix.time(benchmark.1(rowcount)):
rowcount Splus 3.4 Splus 6.0 R
10 .01 0.16 .01
100 .13 1.57 .09
1000 1.29 16.39 .93
10000 15.54 295.93 8.27
The behavior for Splus 6 on this example is around ten times worse
than it used to be with SPlus 3.4 for small numbers, and then gets
worse for larger numbers (i.e. quadratic behavior?)
----------------
Finally, here is my summary of two possible explanations from
Insightful, from when they responded concerning an earlier, different
version of this benchmark (so it is certainly possible that the
explanation does not accurately apply to this benchmark with my new
changes; in that case, the error is mine):
1. When S Version 4 first came out, when a program did this:
List[[i]] <- newValue
it first incremented the reference count on List (including all of its
elements, so this is expensive), then it updates List[[i]], then it
decrements the ref count on List (again expensive). SPlus 6.0 may
still be doing this.
2. A secondary effect (a linear effect) is that more functions
use the somewhat slower new method dispatch system of SV4.
(More functions are generic and more use the new system --
both slow things a bit.) One can use the el() function instead
of "[[" to speed things up since it is not generic (hence
only works with numeric subscripts).
--gary
_______________________________________________________
Gary Sabot | Voice: (781) 647-7776
Delphi Capital Management | FAX: (781) 647-7779
10 Carroll Circle | Internet: gary@sabot.com
Weston, MA 02493 USA |
-------------------------------------------------------
|