Educational systems providing Automated Grading can be very useful for both learner and teacher. After their design and implementation, systems providing automated grading have to be tested and validated, which let the need for real world experiments arise. In this paper we describe an approach to “simulated experiments” that we hope can ease the task of the researchers developing the above mentioned systems. The concept of our proposal is in defining a simulated class by means of statistical distributions of the features that model the student and her/his work in the system. Since such features can be very different from a system to another, we try and keep the description of our framework as general as possible, and define a simulated class by the distribution of the grades that the learners should get (and the system should infer, if correctly working). We show the use of our approach with two systems, that do grading and model the students quite differently. We conclude that our simulation framework can be beneficial in validating an automated grading system, and in allowing to reflect on possible updates of the system, to make its grading more correct.

A protocol for simulated experimentation of automated grading systems / Sterbini, A.; Temperini, M.; Vittorini, P.. - 1241:(2020), pp. 259-270. (Intervento presentato al convegno 10th International Conference in Methodologies and Intelligent Systems for Technology Enhanced Learning, MIS4TEL 2020 tenutosi a L´Aquila; Italy) [10.1007/978-3-030-52538-5_26].

A protocol for simulated experimentation of automated grading systems

Sterbini A.
;
Temperini M.
;
2020

Abstract

Educational systems providing Automated Grading can be very useful for both learner and teacher. After their design and implementation, systems providing automated grading have to be tested and validated, which let the need for real world experiments arise. In this paper we describe an approach to “simulated experiments” that we hope can ease the task of the researchers developing the above mentioned systems. The concept of our proposal is in defining a simulated class by means of statistical distributions of the features that model the student and her/his work in the system. Since such features can be very different from a system to another, we try and keep the description of our framework as general as possible, and define a simulated class by the distribution of the grades that the learners should get (and the system should infer, if correctly working). We show the use of our approach with two systems, that do grading and model the students quite differently. We conclude that our simulation framework can be beneficial in validating an automated grading system, and in allowing to reflect on possible updates of the system, to make its grading more correct.
2020
10th International Conference in Methodologies and Intelligent Systems for Technology Enhanced Learning, MIS4TEL 2020
Automated grading; Machine learning; Peer Assessment
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
A protocol for simulated experimentation of automated grading systems / Sterbini, A.; Temperini, M.; Vittorini, P.. - 1241:(2020), pp. 259-270. (Intervento presentato al convegno 10th International Conference in Methodologies and Intelligent Systems for Technology Enhanced Learning, MIS4TEL 2020 tenutosi a L´Aquila; Italy) [10.1007/978-3-030-52538-5_26].
File allegati a questo prodotto
File Dimensione Formato  
Sterbini_A-Protocol_2020.pdf

solo gestori archivio

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 594.46 kB
Formato Adobe PDF
594.46 kB Adobe PDF   Contatta l'autore

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1542439
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact