Although my original posting was for generating correlated beta variates, I
included the focus of my search was to simulate kappa statistics, which are
used in inter-rater agreement aspects of clinical trials.
I have come across some articles that may be of interest to others. These deal
with the 2 x 2 case of inter-rater agreement, yet the concepts should
generalize to scores with more categories, as in my particular application.
"High Agreement but Low Kappa: I. The Problems of Two Paradoxes", Feinstein
and Cicchetti, Journal of Clinical Epidemeology, 1990, vol 43, No 6, pp 543-549.
"High Agreement but Low Kappa: II. Resolving the Paradoxes", Cicchetti and
Feinstein, Journal of Clinical Epidemeology, 1990, vol 43, No 6, pp 551-558.
"Behavior and interpretation of the kappa statistic: Resolution of the two
paradoxes", Lantz CA and Nebenzahl E. J Clin Epidemiol, 1996, vol 49 pp
431-434.
"Bias, prevalence and kappa", Byrt T, Bishop J and Carlin JB. J Clin
Epidemiol, 1993, vol 46, pp423-429.
"Raking kappa: Describing potential impact of marginal distributions on
measures of agreement", Agresti, Ghosh and Bini, 1995, Biometric Journal, Vol
37, 811-820.
Jim Pratt
>>> "Jim Pratt" <JimP@sonuspharma.com> 10/13/99 05:28PM >>>
I searched the S-News archive and did not find a suggestion for my situation.
I'd like to generate bivariate beta random variables with specified
correlation. Much has been discussed on S-News recently regarding correlated
binary and normal variates, but have not seen anything on correlated beta
variates. Any suggestions would be greatly appreciated.
What I plan to do with these variates:
The kappa statistic for inter-rater agreement 'seems' to behave poorly when the
objects being rated are not uniformly distributed across the entire scoring
scale. E.g., suppose a 4-point scale (0-3) is used, but 75% of the objects are
scored 3, 24% scored 2, .5% each at 0 and 1. Even though there is high
agreement between the two raters (80% of objects are on diagonal), the observed
kappa was 0.30. I am interested in understanding the behavior of kappa in such
non-uniform situations. The plan is to generate bivariate beta scores as
latent variables with specified correlation, collapse to categorical scores,
and assess how observed kappa relates to correlation of the latent variates.
In a series of papers by Cicchetti and Feinstein, they discuss paradoxes of
kappa statistics in the context of 2-point scoring (yes/no; pos/neg). They
discuss the effect of prevalence on the kappa statistics and how severity of
prevalence is related to the maximum possible kappa value that could be
attained. They propose adjusting the computed kappa by weighting by the
maximum possible. [To me, this indicates that kappa statistics should not be
reviewed in isolation, but with some measure of observed prevalence.] I am
looking for an extension of Cicchetti and Feinstein's work to polytomous
scales, and would appreciate any help on that as well.
Will summarize to the group.
Regards,
James Pratt
Sonus Pharmaceuticals
jimp@sonuspharma.com
-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news
-----------------------------------------------------------------------
This message was distributed by s-news@wubios.wustl.edu. To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message: unsubscribe s-news
|