A brief note on the interpretation of the Area Under the ROC:
As Dr. Harrell suggests, in general the Area is a reasonable measure of model
power when the ROC curves do not cross. However, when they do (which often
happens in some applications in finance, for example), the "best" model will
depend on the cost function used to evaluate the model performance, and may be
application specific. This cost function can also be used to identify optimal
cutoff points (if cutoffs are desired) or to determine "pricing." If this is
useful in your work, I discuss some of these issues in the following paper:
Stein, R. M. (2005) The relationship between default prediction and lending
Profits: Integrating ROC analysis and loan pricing, Journal of Banking &
Finance 29. 1213–1236 http://www.moodyskmv.com/research/files/wp/JBF_2026.pdf
~ ~ ~ ~ ~ ~ ~ ~ ~
Roger M. Stein
Moody's Investors Service
99 Church St.
New York, NY 10007
-----Original Message-----
From: s-news-owner@lists.biostat.wustl.edu
[mailto:s-news-owner@lists.biostat.wustl.edu]
Sent: Friday, December 09, 2005 6:20 AM
To: s-news@lists.biostat.wustl.edu
Subject: [s-news] S-News Daily Digest V2 #1293
S-News Daily Digest
Volume 2 : Issue 1293 : "text" Format
Messages in this Issue:
POP Version 3 Released
Re: Function for summarizing the table to two factors and a frequency
locate text in function(s).( summary)
signif.star in s-plus?
ROC curves in S?
Re: ROC curves in S?
----------------------------------------------------------------------
Date: Thu, 08 Dec 2005 11:23:57 +0000
From: Patrick Burns <pburns@pburns.seanet.com>
To: s-news@lists.biostat.wustl.edu
Subject: POP Version 3 Released
Message-ID: <439817CD.5020403@pburns.seanet.com>
Version 3 of POP Portfolio Construction Suite is now
released. This is commercial software that runs under
both S-PLUS and R.
The highlights are:
It is now available under Linux as well as Windows.
More efficient optimization, especially in 'portfolio.optimizer' when there are
more than about 30 assets traded.
More constraints: including portfolio threshold constraints, lower bounds on
variance and tracking error constraints, upper bounds on alpha constraints,
lower bounds on portfolio size and the number of assets traded, cost
constraints, constraints on the number of positions to close. Most, but not
all, of these additional constraints are mainly useful for generating random
portfolios.
Forced trades are now supported.
More details are on the Burns Statistics website, including
the updated POP User's Manual.
Patrick Burns
patrick@burns-stat.com
+44 (0)20 8525 0696
http://www.burns-stat.com
------------------------------
Date: Thu, 8 Dec 2005 07:26:44 -0600
From: David L Lorenz <lorenz@usgs.gov>
To: Jagadish Rangrej <jrangrej@yahoo.com>
Cc: "s-news@lists.biostat.wustl.edu" <s-news@lists.biostat.wustl.edu>
Subject: Re: Function for summarizing the table to two factors and a frequency
Message-ID: <OF3BE6133D.F60FBF29-ON862570D1.004987B1-862570D1.0049DC47@usgs.gov>
Jagadish,
The tapply() function will produce a summary table that you want. Note
that this version return a matrix if no more than 1 value is found for n1 or
n2. The I() function simply returns what is passed to it. Dave
> tapply(Junk$n1, Junk[,1:2], I)
112 114 142 312 342
311 239.87344 166.50672 870.5615 120.39770 1
313 27.75878 19.03609 197.4086 35.53144 NA
341 1346.87779 553.11880 969.2233 NA NA
Jagadish Rangrej <jrangrej@yahoo.com>
Sent by: s-news-owner@lists.biostat.wustl.edu
12/07/2005 04:16 PM
To
"s-news@lists.biostat.wustl.edu" <s-news@lists.biostat.wustl.edu> cc
Subject
[S] Function for summarizing the table to two factors and a frequency
Hello,
is there a generic function which can generate the
summary table (like the one generated by 'table'
function in splus/R) based on two factors and a column
of counts
I have a table of n x 4 out of which the 1st two are
factors and other two are counts for case and control
resp.. I need to generate either a table of 3
dimensions f1 x f2 x 2, or two tables of f1 x f2 size
with there dimnames matched for both controls and
cases.
Here is an example:
mydata =
f1 f2 n1 n2
1 311 112 239.87344 239.829302
2 311 114 166.50672 168.512009
3 311 142 870.56154 869.301191
4 311 312 120.39770 119.280942
5 311 342 1.00000 -1.308982
6 313 112 27.75878 27.834905
7 313 114 19.03609 19.961629
8 313 142 197.40857 198.818765
9 313 312 35.53144 34.929056
10 341 112 1346.87779 1346.927249
11 341 114 553.11880 553.609228
12 341 142 969.22327 969.618471
---------------------------------------
f2
f1 112 114 142 312 342
311 1 1 1 1 1
313 1 1 1 1 0
341 1 1 1 1 1
343 1 1 1 1 0
411 1 1 1 1 0
413 0 0 1 0 0
441 1 1 1 1 0
443 1 0 0 0 0
but instead of '1's in three I need a N1, and N2's.
all help will be greatly appreciated,
Jag
__________________________________________
Yahoo! DSL – Something to write home about.
Just $16.99/mo. or less.
dsl.yahoo.com
--------------------------------------------------------------------
This message was distributed by s-news@lists.biostat.wustl.edu. To unsubscribe
send e-mail to s-news-request@lists.biostat.wustl.edu with the BODY of the
message: unsubscribe s-news
[Attachment of type text/html removed.]
------------------------------
Date: Thu, 8 Dec 2005 06:29:40 -0800 (PST)
From: Kamil Toth <kamiltoth@yahoo.com>
To: "s-news@lists.biostat.wustl.edu" <s-news@lists.biostat.wustl.edu>
Subject: locate text in function(s).( summary)
Message-ID: <20051208142940.69406.qmail@web35811.mail.mud.yahoo.com>
Many thanks to Andrew White, carr.gj@pg.com, and Rich from Mango Solutions
(sorry, not all the full names are available) for the help regarding my
request "locate text in function(s)"
All the suggestions are really smart. The Rich's one-liner works best for me.
Thank you all
Kamil
---------------------------------
Yahoo! Shopping
Find Great Deals on Holiday Gifts at Yahoo! Shopping
[Attachment of type text/html removed.]
------------------------------
Date: Thu, 08 Dec 2005 20:32:41 +0100
From: Henrik Parn <henrik.parn@bio.ntnu.no>
To: s-news@lists.biostat.wustl.edu
Subject: signif.star in s-plus?
Message-ID: <43988A59.2000506@bio.ntnu.no>
Dear S-plus users,
Is there an equivalent of the 'R function' signif.stars in S-plus?
Thanks in advance!
Henrik
------------------------------
Date: Thu, 8 Dec 2005 12:57:37 -0800 (PST)
From: Nicola Koper <nkoper@yahoo.com>
To: S-news <s-news@wubios.wustl.edu>
Subject: ROC curves in S?
Message-ID: <20051208205737.39535.qmail@web36209.mail.mud.yahoo.com>
Is there a way to generate ROC curves in Splus for
logistic regression calculated using GLM? I'm using
version 6.2.
Thanks,
Nicky
************************************************************
Dr. Nicola Koper
Assistant Professor
Natural Resources Institute
University of Manitoba
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
------------------------------
Date: Thu, 08 Dec 2005 15:00:38 -0600
From: Frank E Harrell Jr <f.harrell@vanderbilt.edu>
To: Nicola Koper <nkoper@yahoo.com>
Cc: S-news <s-news@wubios.wustl.edu>
Subject: Re: ROC curves in S?
Message-ID: <43989EF6.6080306@vanderbilt.edu>
Nicola Koper wrote:
> Is there a way to generate ROC curves in Splus for
> logistic regression calculated using GLM? I'm using
> version 6.2.
>
> Thanks,
> Nicky
Don't draw the curve, which can lead people to select arbitrary
cutpoints on the predicted probabilties and to use an improper loss
(utility) function. But the area under the ROC curve is a good
discrimination measure. You get it (C index) by
library(Hmisc,T)
library(Design,T)
f <- lrm(y ~ x1 + x2 + ....)
f # prints C and other rank measures of predictive discrimination
Frank Harerll
>
> ************************************************************
>
> Dr. Nicola Koper
> Assistant Professor
> Natural Resources Institute
> University of Manitoba
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
> --------------------------------------------------------------------
> This message was distributed by s-news@lists.biostat.wustl.edu. To
> unsubscribe send e-mail to s-news-request@lists.biostat.wustl.edu with
> the BODY of the message: unsubscribe s-news
>
--
Frank E Harrell Jr Professor and Chair School of Medicine
Department of Biostatistics Vanderbilt University
------------------------------
End of [s-news] S-News Daily Digest V2 #1293
**********
---------------------------------------
The information contained in this e-mail message, and any attachment thereto,
is confidential and may not be disclosed without our express permission. If
you are not the intended recipient or an employee or agent responsible for
delivering this message to the intended recipient, you are hereby notified that
you have received this message in error and that any review, dissemination,
distribution or copying of this message, or any attachment thereto, in whole or
in part, is strictly prohibited. If you have received this message in error,
please immediately notify us by telephone, fax or e-mail and delete the message
and all of its attachments. Thank you.
Every effort is made to keep our network free from viruses. You should,
however, review this e-mail message, as well as any attachment thereto, for
viruses. We take no responsibility and have no liability for any computer
virus which may be transferred via this e-mail message.
|