Diagnostic agreement in the assignment of the 2017 AAP/EFP world workshop case definitions of periodontitis and peri-implant health and diseases

Introducing a new classification into clinical practice and research requires meticulous planning and a new way of thinking. A learning curve to understand and become acquainted with its novel nature is always needed. Therefore, it was considered of critical importance to assess whether the structure and integrated information of the 2017 AAP/EFP World Workshop Classification of Periodontal and Peri-implant Diseases and Conditions allow for a predictable definition of clinical cases. Consequently, the main objective of this thesis was to determine the level of accuracy and consistency of the clinicians in staging and grading, as well as in the definition of cases of peri-implant health and diseases. The present cumulative dissertation consists of an introduction, three self-contained research papers and the conclusion of the thesis. The introduction (Chapter 1) offers an overview of the topics of interest and the context in which the research was carried out. Furthermore, it outlines the rationale for the development of the present investigation. The three research papers (Chapters 2, 3 and 4) fall naturally into two parts. The first part (Chapters 2 and 3) included two studies focusing on the assessment of the diagnostic agreement in the definition of periodontitis cases using the staging and grading system without the aid of any implementation or with the support of a dedicated software. In the first study (Chapter 2), thirty participants (10 periodontal experts, 10 general dentists, and 10 undergraduate students) and a gold standard examiner (selected among the authors of the 2017 AAP / EFP World Workshop case definitions of periodontitis) were asked to double-evaluate 25 fully documented cases of periodontitis. Fleiss kappa was used to estimate consistency across examiners. Intraclass correlation coefficient (ICC) was used to calculate consistency across time. Quadratic weighted kappa and percentage of complete agreement versus gold standard were computed to assess accuracy. Diagnosis was highly consistent over time. In particular, the highest ICC was provided by students for the stage (0.91), while the lowest ICC was provided by general dentists for the extent (0.79). Case definitions were moderately reliable. Indeed, Fleiss kappa for stage, extent and grade were 0.48, 0.37 and 0.45, respectively. Accuracy was almost perfect for the stage (pair-wise comparisons with the gold standard examiner showed a mean kappa value >0.81) and moderate for grade and extent (pair-wise comparisons with the gold standard examiner showed a mean kappa value >0.41 for grade and extent). Complete agreement with the gold standard examiner for all three components of the case definition was reached in 47.2% of cases. Nevertheless, the study identified specific factors associated with lower consistency and accuracy and recognized the need for further efforts to improve the training of general dentists. In the second study (Chapter 3), the same 10 general dentists from the previous survey were asked to independently assess 25 cases of periodontitis using a software application. Accuracy and consistency were analyzed using the same statistical methods of the earlier study. Supported by the software application, general dentists have reached substantial inter-rater agreement (Fleiss kappa was 0.818, 0.608 and 0.632 for stage, extent and grade, respectively). Assignments of stage and grade were highly accurate. More in detail, pairwise comparisons of each dentist against the reference definition resulted in at least substantial agreement in 100% of cases for stage and in 70% of cases for grade. However, complete case definitions were correctly diagnosed in only 53.6% of cases. Nevertheless, this result represented a 16% increase in accuracy over the previous attempt without any implementation tool. Additionally, the study recognized possible reasons that could lead to decreased accuracy using the software application. The second part (Chapter 4) included one study focusing on evaluating the diagnostic agreement in assigning case definitions of peri-implant health and diseases. Indeed, the third study aimed to evaluate the consistency and accuracy in defining dental implant cases using the 2017 AAP/EFP World Workshop classification. Ten undergraduate students and 10 general dentists and a gold standard examiner (selected among the authors of the 2017 AAP/EFP World Workshop case definitions of peri-implant health and diseases) participated in this study. All examiners were provided with documentation of 25 dental implants including: years since the delivery of the prosthetic reconstruction, clinical (intra-oral photographs, probing depths, bleeding on probing and suppuration on probing) and radiographic data. Eleven out 25 cases were also provided with baseline readings. They were asked to define all cases using the 2017 AAP / EFP World Workshop classification. Reliability among examiners was evaluated using the Fleiss kappa statistics. Accuracy was estimated using quadratic weighted kappa for pairwise comparisons between each rater and the gold standard examiner and percentage of complete agreement. Fleiss kappa for the inter-rater reliability was 0.50, which was interpreted as a moderate agreement. Agreement between each examiner and the gold standard examiner was mostly moderate (mean quadratic weighted kappa value = 0.492). Accurate case definitions were obtained in 55.0% of cases. Absence of longitudinal data impaired agreement with the reference diagnosis (p<0.001). Key elements to be interpreted in order to better discriminate between peri-implant health and peri-implant mucositis and between peri-implant mucositis and peri-implantitis were identified. The conclusion of the thesis (Chapter 5) starts with an overall analysis of the main findings of the three related researches, integrating the results obtained. Moreover, it highlighted some aspects of the 2017 AAP/EFP World Workshop case definitions - recognized by the lack of agreement among examiners and the gold standard diagnosis – which should require further clarification. Finally, the future perspectives in this field are mentioned.