A measure of interrater absolute agreement for ordinal scales is proposed capitalizing on the dispersion index for ordinal variables proposed by Giuseppe Leti. The procedure allows to overcome the limits affecting traditional measures of interrater agreement in different fields of application. An unbiased estimator of the proposed measure is introduced and its sampling properties are investigated. In order to construct confidence intervals for interrater absolute agreement both asymptotic results and bootstrapping methods are used and their performance is evaluated. Simulated data are employed to demonstrate the accuracy and practical utility of the new procedure for assessing agreement. Finally, an application to a real case is provided.

A measure of interrater absolute agreement for ordinal categorical data / Conti, Pier Luigi; Bove, Giuseppe; Marella, Daniela. - In: STATISTICAL METHODS & APPLICATIONS. - ISSN 1613-981X. - (2021). [10.1007/s10260-020-00551-5]

A measure of interrater absolute agreement for ordinal categorical data

Pier Luigi Conti
Secondo
Methodology
;
Giuseppe Bove
Methodology
;
Daniela Marella
Methodology
2021

Abstract

A measure of interrater absolute agreement for ordinal scales is proposed capitalizing on the dispersion index for ordinal variables proposed by Giuseppe Leti. The procedure allows to overcome the limits affecting traditional measures of interrater agreement in different fields of application. An unbiased estimator of the proposed measure is introduced and its sampling properties are investigated. In order to construct confidence intervals for interrater absolute agreement both asymptotic results and bootstrapping methods are used and their performance is evaluated. Simulated data are employed to demonstrate the accuracy and practical utility of the new procedure for assessing agreement. Finally, an application to a real case is provided.
2021
ordinal data; resampling; interrater agreement
01 Pubblicazione su rivista::01a Articolo in rivista
A measure of interrater absolute agreement for ordinal categorical data / Conti, Pier Luigi; Bove, Giuseppe; Marella, Daniela. - In: STATISTICAL METHODS & APPLICATIONS. - ISSN 1613-981X. - (2021). [10.1007/s10260-020-00551-5]
File allegati a questo prodotto
File Dimensione Formato  
Bove_measure-of-interrater_2020.pdf

accesso aperto

Tipologia: Documento in Post-print (versione successiva alla peer review e accettata per la pubblicazione)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 390.88 kB
Formato Adobe PDF
390.88 kB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1465979
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? 2
social impact