Performance of the 2017 and 2010 Revised McDonald Criteria in Predicting MS Diagnosis After a Clinically Isolated Syndrome

Background and Objectives To compare the performance of the 2017 revisions to the McDonald criteria with the 2010 McDonald criteria in establishing multiple sclerosis (MS) diagnosis and predicting prognosis in patients with clinically isolated syndrome (CIS) suggestive of MS. Methods CSF examination and brain and spinal cord MRI obtained ≤5 months from CIS onset and a follow-up brain MRI acquired within 15 months from CIS onset were evaluated in 785 patients with CIS from 9 European centers. Date of second clinical attack and of reaching Expanded Disability Status Scale score (EDSS) ≥3.0, if they occurred, were also collected. Performance of the 2017 and 2010 McDonald criteria for dissemination in space (DIS), dissemination in time (DIT) (including oligoclonal bands assessment), and DIS plus DIT for predicting a second clinical attack (clinically definite MS [CDMS]) and EDSS ≥3.0 at follow-up was evaluated. Time to MS diagnosis for the different criteria was also estimated. Results At follow-up (median 69.1 months), 406/785 patients with CIS developed CDMS. At 36 months, the 2017 DIS plus DIT criteria had higher sensitivity (0.83 vs 0.66), lower specificity (0.39 vs 0.60), and similar area under the curve values (0.61 vs 0.63). Median time to MS diagnosis was shorter with the 2017 vs the 2010 or CDMS criteria (2017 revision, 3.2; 2010 revision, 13.0; CDMS, 58.5 months). The 2 sets of criteria similarly predicted EDSS ≥3.0 milestone. Three periventricular lesions improved specificity in patients ≥45 years. Discussion The 2017 McDonald criteria showed higher sensitivity, lower specificity, and similar accuracy in predicting CDMS compared to 2010 McDonald criteria, while shortening time to diagnosis of MS. Classification of Evidence This study provides Class II evidence that the 2017 McDonald Criteria more accurately distinguish CDMS in patients early after a CIS when compared to the 2010 McDonald criteria.


Discussion
The 2017 McDonald criteria showed higher sensitivity, lower specificity, and similar accuracy in predicting CDMS compared to 2010 McDonald criteria, while shortening time to diagnosis of MS.

Classification of Evidence
This study provides Class II evidence that the 2017 McDonald Criteria more accurately distinguish CDMS in patients early after a CIS when compared to the 2010 McDonald criteria.
In 2001, MRI was formally included in the criteria for multiple sclerosis (MS) diagnosis to demonstrate dissemination in space (DIS) and dissemination in time (DIT) of MS lesions. 1 Subsequent iterations of the McDonald criteria 2,3 have simplified MS diagnosis, while maintaining sensitivity, specificity, and accuracy. The McDonald criteria shorten the time from onset of symptoms to MS diagnosis in patients with a clinically isolated syndrome (CIS) suggestive of MS, without waiting for the occurrence of a second clinical event. 4 This is crucial because an increasing number of therapies have been demonstrated to be effective in favorably modifying MS disease course especially if started early. 5 Recently, new evidence regarding the utility of MRI and CSF analysis for MS diagnosis has become available. The MAGNIMS network proposed modified MRI criteria for DIS in 2016. 6 The performance of these evidence-based recommendations was evaluated in a large multicenter MAGNIMS study 7 that informed the 2017 revisions to MS diagnostic criteria, 8 including removing the distinction between symptomatic and asymptomatic lesions and combining cortical and juxtacortical lesions in DIS (eTable 1, data available from Zenodo, doi.org/10.5281/zenodo. 5566178). In addition, in patients with CIS fulfilling DIS criteria, the presence of CSF-specific oligoclonal bands (OCBs) allows a diagnosis of MS in the absence of clinical or MRI evidence of DIT (eTable 1, data available from Zenodo, doi.org/10.5281/zenodo.5566178). 8 An additional modification proposed by the MAGNIMS network 6 to improve the criteria specificity, 7 but not included in the 2017 revisions, was to increase the number of lesions needed to establish periventricular involvement from 1 to 3.
After their publication, several studies compared the performance of the 2017 and 2010 revisions to the McDonald criteria in adult [9][10][11][12][13][14] and pediatric patients with CIS. [15][16][17] Globally, they showed that the 2017 McDonald criteria have higher sensitivity but lower specificity compared with the 2010 criteria in predicting clinically definite MS (CDMS), but a shorter time to MS diagnosis.
Several aspects related to the performance of the 2017 revision of the McDonald criteria still need to be fully evaluated. The performance of these criteria has mainly been assessed in small, monocentric cohorts of patients with CIS. Different outcomes have been used, including the estimation of CDMS after a relatively short or heterogeneous follow-up, or by combining clinical (CDMS) with MRI (new lesions) status. In addition, some important aspects have been only partially explored. These include the influence of age and CIS topography on criteria performance and the effect of treatment initiation on the risk of developing CDMS. The ability of the 2017 revisions to predict disability accumulation has not yet been investigated.
To clarify all these important aspects, we compared the performance of the 2017 McDonald criteria in predicting CDMS development and MS prognosis, and in enabling an earlier MS diagnosis, with that of the 2010 McDonald criteria, in a large multicenter study of patients with a typical CIS suggestive of MS. The influence of type of CIS onset and increasing the number of periventricular lesions needed to demonstrate DIS from 1 to 3 (which might have a different relevance according to age at onset) was also evaluated.

Methods
Standard Protocol Approvals, Registrations, and Patient Consents Approval was received from the institutional ethical standards committee on human experimentation of each participating center for any experiments using human subjects. Written informed consent was obtained from all patients participating in the study at the time of data acquisition. Glossary aHR = adjusted hazard ratio; AUC = area under the curve; CDMS = clinically definite multiple sclerosis; CI = confidence interval; CIS = clinically isolated syndrome; DE = dual-echo; DIR = double inversion recovery; DIS = dissemination in space; DIT = dissemination in time; DMT = disease-modifying treatment; EDSS = Expanded Disability Status Scale; FLAIR = fluidattenuated inversion recovery; Gd = gadolinium; MS = multiple sclerosis; OCB = oligoclonal band; STIR = short tau inversion recovery; WM = white matter.
The study design was similar to that of previous studies aimed at assessing the performance of MS diagnostic criteria. 7,18 Centers identified patients with CIS recruited into local prospective clinical and MRI follow-up studies from June 1995 to October 2020 with (1) age between 18 and 60 years; (2) a diagnosis of CIS suggestive of MS 19 ; (3) a typical clinical presentation of relapsing-onset MS 20 ; (4) a complete neurologic examination, with scoring of the Expanded Disability Status Scale (EDSS), performed within 5 months from the clinical onset; (5) baseline brain and spinal cord MRI scans obtained within 5 months from the clinical onset; (6) a follow-up brain MRI obtained ≤15 months from CIS onset. Development of CDMS was defined as the occurrence of a second clinical event attributable to demyelination lasting more than 24 hours and after an interval ≥1 month from the first attack, with evidence of 2 separate lesions. Time to CDMS was calculated as the interval between the first 2 clinical events.
The following information was collected: age at CIS onset, sex, presence of CSF-specific OCBs (where available), date and topography of CIS onset, dates of the second event and of last available follow-up, date of reaching EDSS ≥3.0, date of initiation and type of disease-modifying treatment (DMT), dates of MRI scans, and field strengths of the scanners used.

MRI Analysis and CSF Examination
For the brain, axial dual-echo (DE) or fast fluid-attenuated inversion recovery (FLAIR) and postcontrast (0.1 mmol/kg gadolinium [Gd]-DTPA; acquisition delay ≈10 minutes) T1weighted sequences were acquired at baseline and at followup. Slice thickness varied between 3 and 5 mm, in-plane resolution between 0.45 and 1.0 mm, no gap between slices. A brain double inversion recovery (DIR) sequence was available for 337/785 (42.9%) patients with CIS from 3 centers (Barcelona, Belgrade, and Milan). For the spinal cord, sagittal short tau inversion recovery (STIR) or T2-weighted and postcontrast T1-weighted sequences (0.1 mmol/kg Gd-DTPA; acquisition delay 5 minutes) with 3 mm slice thickness, in-plane resolution between 0.4375 and 1.0 mm, no gap between slices, covering the cervical and thoracic cord were acquired. For 120/785 (15.3%) patients with CIS, baseline spinal cord involvement was established using available MRI reports, due to missing or corrupted raw MRI data.
All images were assessed by consensus by 2 experienced observers (P.P. and M.A.R.), blinded to the patients' identity and MS status at the Neuroimaging Research Unit (Milan, Italy). Brain white matter (WM) lesions were identified on DE/ FLAIR images and were defined as hyperintensities involving at least 3 voxels, present on at least 2 slices, and visible on 2 different sequences (e.g., FLAIR and T2 or proton density and T2). Spinal cord lesions were identified on sagittal STIR or T2-weighted sequences. Total number of WM lesions, number of periventricular (abutting the lateral ventricles without intervening WM), juxtacortical (touching the cortex), cortical (within the cortex), posterior fossa (located in the brainstem, cerebellar peduncles, and cerebellar hemispheres), and spinal cord lesions were evaluated following published recommendations. 21 Cortical lesions and juxtacortical lesions identified from DIR (available in 337/785 [43%]) and T2 and FLAIR sequences were combined. 8 Gd-enhancing lesions (area of hyperintensity on postcontrast T1-weighted images) 21 were identified on postcontrast T1-weighted scans.
On the MRI scan acquired at follow-up, the numbers of new T2-hyperintense and Gd-enhancing lesions were quantified.
OCBs were evaluated in 670/785 (85%) patients with CIS in the CSF and serum at the time of baseline clinical evaluation by agarose isoelectric focusing combined with immunoblotting. When bands were present only in the CSF, they were considered CSF-specific.

DIS and DIT Criteria
The 2010 3 and 2017 8 McDonald DIS criteria were assessed on baseline MRI scans; the 2010 3 and 2017 8 McDonald DIT criteria were defined on baseline and follow-up MRI and according to CSF-specific OCB status (eTable 1, data available from Zenodo, doi.org/10.5281/zenodo.5566178). Although presence of OCBs does not reflect DIT, but is an alternative to DIT, we used the definition of DIT including OCBs. The fulfilment of DIS plus DIT criteria for both 2010 and 2017 was also assessed.
The performance of the 2017 McDonald criteria 8 was evaluated also without OCB assessment.
As additional analyses, we compared the performances of 2017 and 2010 McDonald criteria according to the type of CIS at onset (i.e., optic neuritis, brainstem/cerebellar syndrome, or spinal cord syndrome). The performance in patients with CIS with a hemispheric or multifocal onset was not evaluable due to their limited number. Finally, we evaluated criteria performance using 3 instead of 1 periventricular lesions to define periventricular involvement, according to age at onset (<25, 25-34, 35-44, and ≥45 years).

Statistical Analysis
Cumulative/dynamic time-dependent receiver operating characteristic curve analysis 22 for censored survival data was applied to assess the performance of the MRI criteria for DIS, DIT, and DIS plus DIT (also without OCB assessment), using the clinical status (CDMS or CIS) over time as outcome. Sensitivity, specificity, accuracy, and positive and negative predictive values at months 36 and 60 were calculated. Bias-corrected and accelerated bootstrap method 23 was used to estimate 95% confidence intervals (CIs). Analyses were repeated after excluding all patients with CIS receiving DMT before the second clinical event.
The cumulative risk of CDMS development from the first clinical event to the last available follow-up was represented using Kaplan-Meier survival curves (patients censored according to their follow-up). Extended Cox regression models using time to CDMS as the outcome and adjusted for age (continuous), sex (binary), treatment (binary, time-dependent, i.e., treatment effects were modeled considering the time when a patient started any treatment [thus considering 2 conditions: without or under treatment]), and disease onset type (optic neuritis, brainstem/cerebellar, spinal cord, hemispheric, multifocal), were performed to obtain adjusted hazard ratios (aHRs) and 95% CIs. A shared gamma-frailty term was also included to address center effects, accounting for unobserved heterogeneity and statistical dependence between clustered time-to-event data. 24 The possible interactions between MRI criteria and treatment or type of onset were explored by estimating similar models including a specific interaction term.
Similar models were also estimated using time to reach EDSS ≥3.0 as the outcome.
Median time to MS diagnosis according to different criteria (i.e., 2017 or 2010 McDonald criteria or CDMS) were estimated using Kaplan-Meier survival curves.

Data Availability
The corresponding author had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. The anonymized dataset used and analyzed during the current study are available from the corresponding author on reasonable request.

Demographic, Clinical, and MRI Data
From 958 patients with CIS, the final cohort comprised 785 patients with CIS fulfilling inclusion criteria. Figure 1 shows a study flow chart and Table 1 summarizes the main baseline demographic, clinical, and MRI findings of these patients.         Zenodo, doi.org/10.5281/zenodo.5566178). These results were confirmed when considering CDMS at month 60 as the outcome, although these differences were marginal for DIS plus DIT (eFigure 1 and eTable 4, data available from Zenodo, doi. org/10.5281/zenodo.5566178).

Prediction of CDMS and EDSS ≥3
Although the cumulative risk of CDMS development was similar for the different sets of criteria, the lack of fulfilment of the 2017 McDonald criteria was associated with a higher risk of nonconverting to CDMS when compared to the 2010 McDonald criteria (Figure 2 and Table 2   criteria and treatment status was statistically significant (p ranging from <0.001 to 0.009), with aHRs being lower under vs without treatment (eTable 5, data available from Zenodo, doi.org/10.5281/zenodo.5566178).

Discussion
By evaluating a large multicenter cohort of patients with typical CIS, 8 our study demonstrated that the 2017 McDonald criteria had higher sensitivity, lower specificity, and overall similar accuracy in predicting conversion to CDMS than the 2010 McDonald criteria. This validation study extends previous studies, 9,12,14 which are characterized by high between-study variability performance of the different criteria, possibly due to the heterogeneity in the demographic and clinical characteristics of the patients evaluated, the MRI protocols used to define MRI criteria, lengths of the follow-ups, statistical approaches used, and the influence of treatment.
We confirmed that the inclusion of OCB assessment 25 increased the sensitivity, reducing the specificity, while preserving the accuracy of the criteria. The decreased specificity derived from OCB evaluations, not done in all cases but requested according to local protocols, could raise some concerns regarding the risk of MS overdiagnosis. 26 Our analysis also showed that although the performance of the 2017 McDonald criteria seems slightly worse in the short term than the 2010 McDonald criteria, due to a lower specificity, the overall accuracy increases over time, thus the presence of OCBs contributes to correctly identify patients developing CDMS when longer follow-ups are considered, underlying the relevance of longer evaluation to better evaluate the performance of the diagnostic criteria. The progressive improvement of criteria performance with time could be due to the effects of DMTs that, if started, may delay the occurrence of a second event, thus negatively influencing the specificity of the criteria when the follow-up is limited to a few years.
The performance of the 2017 and 2010 McDonald criteria was similar in patients with CIS with optic neuritis. This is expected, also considering the limited relevance in distinguishing symptomatic from asymptomatic lesions in patients with optic neuritis, with the slight differences mainly due to the inclusion of OCB assessment. Accordingly, the evaluation of optic nerve involvement could improve the diagnostic accuracy in this type of onset. 6,7,27,28 Interestingly, considering especially the pointwise evaluation at month 36 more than that at month 60, and the overall accuracy over time, it seems clear that, compared to the 2010 McDonald criteria, the 2017 McDonald criteria showed better performance in predicting CDMS development in patients with CIS with a brainstem/cerebellar syndrome. The inclusion of infratentorial lesions irrespective of being symptomatic in patients with this type of CIS allows capture of the contribution of the infratentorial site, which is known to be clinically relevant, in respect to CDMS development 29,30 and  Periventricular lesions increase with age, especially in people with cerebrovascular risk factors, 35 and occur in several neurologic conditions characterized by WM lesions and mimicking MS, including small vessel disease, 36 and migraine. [37][38][39] In previous studies the requirement for 3 periventricular lesions improved specificity of the 2010 7,40,41 and 2017 12 McDonald criteria for DIS, especially in older patients with CIS. 41 In addition, a threshold of 3 or more periventricular lesions was found to be one of the most accurate predictors of CDMS conversion. 29,42,43 In line with this, we found that 3 periventricular lesions improved the specificity and accuracy of the 2017 McDonald DIS criteria, but slightly reduced sensitivity, especially in patients with CIS aged ≥45 years. The main aim of the subsequent iterations of the McDonald criteria is to allow an earlier and more accurate MS diagnosis in people who present with a CIS, but it remains essential to reduce the risk of misdiagnosis due to a combination of increased sensitivity and reduced specificity, that is, an oversimplification of MS criteria. 26 Among the proposed modifications, the evaluation of 3 periventricular lesions appears easily implementable in the clinical setting and may present a distinctive criterion to improve the specificity and the accuracy of the diagnostic criteria at least in older patients with CIS who do not satisfy DIT criteria, as suggested by a previous study, 41 or patients with comorbidities.
Survival analyses showed that the cumulative risk of CDMS development was similar in patients with CIS fulfilling the criteria independently from the set of criteria investigated. Conversely, patients with CIS who did not fulfil the 2017 McDonald criteria had a higher risk of not developing CDMS than patients who met the criteria. These findings, combined with those of a previous study, 9 suggest that the 2017 McDonald criteria could be useful to identify patients with CIS at low risk of developing a second relapse.
Of note, the significant interaction found between the different criteria and treatment status, with lower aHR found under treatment, confirms that the use of DMTs worsens the performance of the criteria and can explain, at least in part, differences in the performances of the criteria compared with previous reports. 18,40,44,45 This hypothesis is also confirmed by the sensitivity analyses performed excluding all patients with CIS starting a DMT before the occurrence of a second clinical event that showed an improvement of the specificity of both the 2017 and 2010 revisions of the McDonald criteria compared to the analyses that evaluated the whole cohort of patients with CIS included in the study.
Regarding the prognostic value for disability accumulation, survival probability analyses showed that patients with CIS fulfilling either the 2010 or the 2017 McDonald criteria had a significantly higher risk to reach an EDSS ≥3.0 than patients who did not meet the criteria, and that the 2 sets of criteria had similar performance, with a slight superiority of the 2017 McDonald criteria. This is in line with previous studies 45 and supports the relevance of infratentorial, 31,32 spinal cord, 33,34,46 and Gd-enhancing lesions 46 but also of CSF-specific OCBs [47][48][49] as relevant predictors of clinical disability.
Finally, consistent with previous studies, 9,11,13,50 5 Further studies are needed to demonstrate that treatment start after clinical onset instead of waiting until a second clinical relapse may also positively limit longterm disability progression. 5 Clearly, this aspect could be negatively counterbalanced by the risk of misdiagnosis. 26 It should be noted that the apparent lower specificity of the 2017 McDonald criteria could reflect earlier treatment with a lower chance of developing DIT, calling into question the appropriateness of the CDMS outcome. It should be also noted that, although lower at month 36, specificity of the 2017 revisions of the McDonald criteria increased at month 60 (0.46) and further improved at month 120 (0.53), approaching that of the 2010 McDonald criteria (0.62), but still remaining lower.
Nevertheless, the 2017 McDonald criteria must be applied in the right context, after the exclusion of differential diagnoses, in patients who present with symptoms and signs typical of MS 26 and with the proper assessment of lesions on MRI. 21 A lower specificity may determine a higher prevalence of MS misdiagnosis and unnecessary initiation of DMT may be associated with unnecessary risks and morbidity in misdiagnosed patients. 26 This study has some limitations. First, the analyses have been performed retrospectively. Despite this, all patients with CIS included are part of ongoing prospective studies performed by each participating center, allowing us to have long follow-up to better investigate the performance of the different sets of criteria. Second, patients with CIS were collected from highly specialized centers, thus possibly selecting patients with higher lesion number and risk of CDMS conversion (52% in our cohort). However, the multicenter setting with MRI examinations acquired with both 1.5T and 3.0T and different sequence parameters allowed us to evaluate the MRI criteria in a cohort of patients with CIS that should be representative of the European clinical scenario. Third, cortical lesion and OCB assessment were not available for all patients. However, cortical lesion evaluation was found to not significantly contribute to DIS criteria performance, 7 whereas OCB assessment was missing only in a minority (2.2%) of patients with CIS not fulfilling DIT criteria. Fourth, no formal statistical testing has been performed between the performances of the 2017 and 2010 revisions to the McDonald criteria and no adjustment for multiplicity was applied. Accordingly, our observations, particularly the subgroup analyses, should be regarded as exploratory and require replication in an independent dataset.
Overall, this study confirms that the 2017 McDonald criteria have higher sensitivity, lower specificity, and overall similar accuracy compared with the 2010 McDonald criteria in predicting CDMS development independently from the type of clinical onset. These criteria simplify the clinical use of MRI criteria without reducing accuracy and allow an earlier diagnosis of MS. Three periventricular lesions should be considered in future revisions of the McDonald criteria to improve specificity and accuracy in older patients with CIS.

Acknowledgment
The funding sources had no role in study design, collection, analysis, or interpretation of data; writing of the report; or the decision to submit the paper for publication. Prof. Ciccarelli is a NIHR Research Professor. Prof. Ciccarelli and Prof. Barkhof receive funding from the NIHR UCLH Biomedical Research Centre.

Study Funding
Prof. Ciccarelli and Prof. Barkhof receive funding from the NIHR UCLH Biomedical Research Centre.