On Sun, 25 Jul 1999, Pat Burns wrote:
> Bruce McCullough wrote:
>
> > > While trying to fit a nonlinear model using nls( ), I get an error
> message
> > > that says I have a singular gradient matrix. I am not sure what causes
> > > this problem. Should I try different initial values of the parameters,
> or
> > > is the problem something else?
> >
> > I have found that supplying analytic first derivatives
> > using deriv() can eliminate this problem (not always,
> > but sometimes). Supplying analytic derivatives
> > is well-described in V&R.
> >
> >
>
> I'll add a few lines of my partial ignorance. In addition to the advise
> thatBill Venables and Doug Bates (and others?) have given, there is another
> approach.
>
> The optimization can be done by a more robust method such as a genetic
> algorithm. This is "robust" in the computational sense, not the statistical
> sense. A genetic algorithm can handle non-differentiability, and multiple
> local minima. There is a crude implementation of a genetic algorithm in
> S that is given in S Poetry.
Um. Yes, of course one could use a better optimizer, but I doubt if a
`genetic algorithm' comes into that category. Of course there are so many
`genetic algorithms' that the possibilities are unbounded, but in hard
non-linear least-squares problems (such as neural networks) they have been
tried very seriously, and seem not to be competitive. As another example,
there is a `genetic algorithm' used in the S-PLUS code for ltsreg (by one
P. Burns, I believe) and that seems to do nowhere near as well as the
standard (in that field) random search algorithms in my lqs function (in
the V&R3 library set). (`nowhere near as well' means many times slower
to achieve a larger value in a minimization problem.)
Beware of `can handle ... multiple local minima'. In what sense? There are
no known computationally feasible global optimizers for many routine
non-linear least-squares problems.
> In this setting, the use of the genetic
> algorithm would be to give a very good starting value to a more
> traditional optimizer.
An informed random multi-start approach may be as good at that.
> However, it is my experience that it is often the case that severe problems
> with convergence means that the data don't fit the model very well. It
> may be worth having a think about the suitability of the model from time
> to time when convergence is a problem.
Alternatively, the model may fit well only for a very narrow range of
parameters (e.g. neural networks), and the problem is to locate that range.
Yet another potential problem is rounding due to single-precision
calculations.
It is rather hard to beat conventional optimization algorithms that make
use of first derivatives. It is quite easy to beat Gauss-Newton methods
(as used by nls) as terms they drop from the Hessian can be very
significant.
--
Brian D. Ripley, ripley@stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272860 (secr)
Oxford OX1 3TG, UK Fax: +44 1865 272595
-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news
|