Hiya there,
I'm using Kappa values to look at interrater reliability for a psychological
tool and I am having a few problems.
The tool uses yes/no questions and thus I have problems with Kappa scores not
reflecting very good levels of agreement when many of the rating scores are
Yes and there are few no's. Any suggestions on playing around with the
values? I thought of keeping the yes/no ratio the same and just getting the
total yes and no scores to be about 50% each of all the scores.This seems to
get a more realistic Kappa value, but I'm not sure if it is allowed?
many thanks
Neil Greenberg
|