Detecting the achievements in INVALSI National Surveys of students who followed educational innovations: a five steps method

The present paper proposes a method to isolate as much as possible the effects of the different educational innovations present in the Italian territory on the learning outcomes measured by the Invalsi National Surveys, to contribute to the debate on the effectiveness of such alternative approaches, such as “Avanguardie educative”, Montessori, “Senza Zaino”, “We Debate”, “Book in Progress”, etc. To illustrate the proposal, we take as an example the research that is taking place on Montessori teaching, in the framework of the PhD in Social & Developmental Psychology and Educational Research at Sapienza University of Rome. According to Cook, Campbell and Peracchio’s (1990) classification, the research design is a design with non-equivalent groups, with only post-tests,because there is no random assignment to experimental and control groups: the assignment takes place with the simple matching (precision control) method, which guarantees the greatest possible isolation of the effect of the independent variable, which keeps under control a) the effects of all the variables that we know affect dependent variables, and b) the effect of their interaction. The proposed method consists in five steps. The first step is about reconstructing the list of schools that follow a certain educational innovation. In the case of Montessori pedagogy, the list of 50 fifth primary classes active in Italy in 2013 and 2016 has been defined. The second step consists in reconstructing the school careers of the students attending the schools on the list. In the case of Montessori schools, for about half of the students of both cohorts, the number of years of Montessori school attended was defined through a meticulous collection of data from the secretariats of 12 Montessori schools in Italy. By this way, in addition to the experimental group of all those enrolled in the Montessori sections identified, there is an experimental group whose number of years of Montessori school attended is also known. The third step is about requesting, to the Statistical Office of Invalsi, the achievement in National Surveys of the students attending the list of identified classes. In the case of Montessori schools, on 24 November 2020 the list of sections and SIDI codes of students whose career was reconstructed was submitted, and between January and February 2021 the databases of the 2013 and 2016 National Surveys was returned, with the distinction between Montessori and non-Montessori students. The fourth step consists in defining the experimental group, made up in this case by the students who attended a Montessori school, and the control group, whose members are identified with the simple matching method (Bailey, 1982, p. 340). The last step consists in verifying, by ANOVA, any statistically significant differences between the distributions of the scores of the experimental groups and related control groups.