A service provided by the WU Library and the WU IT-Services

Significance Tests for the Measure of Raw Agreement

von Eye, Alexander and Mair, Patrick and Schauerhuber, Michael (2006) Significance Tests for the Measure of Raw Agreement. Research Report Series / Department of Statistics and Mathematics, 42. Department of Statistics and Mathematics, WU Vienna University of Economics and Business, Vienna.

[img]
Preview
PDF
Download (356Kb) | Preview

Abstract

Significance tests for the measure of raw agreement are proposed. First, it is shown that the measure of raw agreement can be expressed as a proportionate reduction-in-error measure, sharing this characteristic with Cohen's Kappa and Brennan and Prediger's Kappa_n. Second, it is shown that the coefficient of raw agreement is linearly related to Brennan and Prediger's Kappa_n. Therefore, using the same base model for the estimation of expected cell frequencies as Brennan and Prediger's Kappa_n, one can devise significance tests for the measure of raw agreement. Two tests are proposed. The first uses Stouffer's Z, a probability pooler. The second test is the binomial test. A data example analyzes the agreement between two psychiatrists' diagnoses. The covariance structure of the agreement cells in a rater by rater table is described. Simulation studies show the performance and power functions of the test statistics. (author's abstract)

Item Type: Paper
Keywords: rater agreement / raw agreement coefficient / Stouffer's Z / binomial test / Monte Carlo Simulation
Divisions: Departments > Finance, Accounting and Statistics > Statistics and Mathematics
Depositing User: Repository Administrator
Date Deposited: 03 Nov 2006 12:32
Last Modified: 05 Jul 2015 15:56
URI: http://epub.wu.ac.at/id/eprint/1336

Actions

View Item