Bridging the experience gap in prostate multiparametric magnetic resonance imaging using artificial intelligence: A prospective multi-reader comparison study on inter-reader agreement in PI-RADS v2.1, image quality and reporting time between novice and expert readers

Purpose: The aim of the study was to determine the impact of using a semi-automatic commercially available AI-assisted software (Quantib® Prostate) on inter-reader agreement in PI-RADS scoring at different PI-QUAL ratings and grades of reader confidence and on reporting times among novice readers in multiparametric prostate MRI. Methods: A prospective observational study, with a final cohort of 200 patients undergoing mpMRI scans, was performed at our institution. An expert fellowship-trained urogenital radiologist interpreted all 200 scans based on PI-RADS v2.1. The scans were divided into four equal batches of 50 patients. Four independent readers evaluated each batch with and without the use of AI-assisted software, blinded to expert and individual reports. Dedicated training sessions were held before and after each batch. Image quality rated according to PI-QUAL and reporting times were recorded. Readers' confidence was also evaluated. A final evaluation of the first batch was conducted at the end of the study to assess for any changes in performance. Results: The overall kappa coefficient differences in PI-RADS scoring agreement without and with Quantib® were 0.673 to 0.736 for Reader 1, 0.628 to 0.483 for Reader 2, 0.603 to 0.292 for Reader 3 and 0.586 to 0.613 for Reader 4. Using PI-RADS ≥ 4 as cut-off for biopsy, the AUCs with AI ranged from 0.799 (95 % CI: 0.743, 0.856) to 0.820 (95 % CI: 0.765, 0.874). Inter-reader agreements at different PI-QUAL scores were higher with the use of Quantib, particularly for readers 1 and 4, with Kappa coefficient values showing moderate to slight agreement. Conclusion: Quantib® Prostate could potentially be useful in improving inter-reader agreement among less experienced to completely novice readers if used as a supplement to PACS.