Statistical inference for indices of interrater agreement based on a homogeneity index

The interrater agreement of ratings on a nominal scale regarding a group of targets (individuals, objects, etc.) is usually computed by kappa-type indices. Recently, a homogeneity index for a qualitative variable was considered to define a single target measure of agreement between raters for each target, and to propose a global measure of agreement for the whole group of targets allowing to overcome limitations affecting traditional kappa-type indices. In this paper, the sampling properties and the asymptotic distribution of the proposed index are investigated. Finally, a simulation study is performed to demonstrate the accuracy of the index and an application to real educational data is provided.