s-news
[Top] [All Lists]

Re: ROC curves in S?

To: "Eric C. Lanes" <Lanesec@michigan.gov>
Subject: Re: ROC curves in S?
From: Frank E Harrell Jr <f.harrell@vanderbilt.edu>
Date: Fri, 09 Dec 2005 10:57:45 -0500
Cc: nkoper@yahoo.com, s-news@wubios.wustl.edu
In-reply-to: <s39966cc.076@michigan.gov>
References: <s39966cc.076@michigan.gov>
User-agent: Debian Thunderbird 1.0.2 (X11/20050602)
Eric C. Lanes wrote:
Nicola,

I'll try to be as helpful as I can here and not confuse what is a relatively 
simple request on your part:

I am not an S-Plus user but am aware of the many similarities between this and 
R. I do not profess to know at this point if there is a true S-plus equivalent 
for the ROCR package for R.

You may draw the ROC curve if you like just to eyeball it. Its relative "roughness" or "smoothness" may at least give you a visual of just how varied, explempary or non-exemplary your cases are probability-wise when, for example, a multivariate model (y~x1+x2....xK) has been used. That is, how "consistently" cases "behave" in terms of the spread of predicted probabilities and their "distance" from .50 when applying a particular model or when comparing models. The AUC ("C" in lrm) is merely a summary measure of this "behavior".
Harrell has rightly noted in response to your query the problems associated 
with the arbitrary SELECTION of cutpoints on predicted probabilities, etc.

In R, the ROCR package generates ROC/AUC, false-positive, true-positive, error 
in prediction and a good number of other measures associated with the 
classification task (see documentation .pdf  on CRAN for the list).

The use of lrm for logistic models is fine (also if you have a polytomous y). 
However, there are other procedures which I find are more flexible 
attirubute-wise for the binary outcome y (depending upon what you'd like to 
do): logistf for one, GLM is another.


One minor correction: lrm does not fit polytomous (nominal; multimonial) models but rather ordinal models (proportional odds or continuation ratio models). -Frank

. . .


<Prev in Thread] Current Thread [Next in Thread>