Scientific experiments and robotic competitions share some common traits that can put the debate about developing better experimental methodologies and replicability of results in robotics research on more solid ground. In this context, the Robot Competitions Kick Innovation in Cognitive Systems and Robotics (RoCKIn) project aims to develop competitions that come close to scientific experiments, providing an objective performance evaluation of robot systems under controlled and replicable conditions. In this article, by further articulating replicability into reproducibility and repeatability and by considering some results from the 2014 first RoCKIn competition, we show that the RoCKIn approach offers tools that enable the replicability of experimental results.
Competitions for benchmarking: task and functionality scoring complete performance assessment / Amigoni, Francesco; Bastianelli, Emanuele; Berghofer, Jakob; Bonarini, Andrea; Fontana, Giulio; Hochgeschwender, Nico; Iocchi, Luca; Kraetzschmar, Gerhard; Lima, Pedro; Matteucci, Matteo; Miraldo, Pedro; Nardi, Daniele; Schiaffonati, Viola. - In: IEEE ROBOTICS AND AUTOMATION MAGAZINE. - ISSN 1070-9932. - STAMPA. - 22:3(2015), pp. 53-61. [10.1109/MRA.2015.2448871]
Competitions for benchmarking: task and functionality scoring complete performance assessment
BASTIANELLI, EMANUELE
;IOCCHI, Luca
;NARDI, Daniele
;
2015
Abstract
Scientific experiments and robotic competitions share some common traits that can put the debate about developing better experimental methodologies and replicability of results in robotics research on more solid ground. In this context, the Robot Competitions Kick Innovation in Cognitive Systems and Robotics (RoCKIn) project aims to develop competitions that come close to scientific experiments, providing an objective performance evaluation of robot systems under controlled and replicable conditions. In this article, by further articulating replicability into reproducibility and repeatability and by considering some results from the 2014 first RoCKIn competition, we show that the RoCKIn approach offers tools that enable the replicability of experimental results.| File | Dimensione | Formato | |
|---|---|---|---|
|
Amigoni_Competitions-for-Benchmarking_2015.pdf
solo gestori archivio
Tipologia:
Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza:
Tutti i diritti riservati (All rights reserved)
Dimensione
1.88 MB
Formato
Adobe PDF
|
1.88 MB | Adobe PDF | Contatta l'autore |
|
Amigoni_preprint_Competitions-for-Benchmarking_2015.pdf
accesso aperto
Note: http://dx.doi.org/10.1109/MRA.2015.2448871
Tipologia:
Documento in Pre-print (manoscritto inviato all'editore, precedente alla peer review)
Licenza:
Tutti i diritti riservati (All rights reserved)
Dimensione
1.48 MB
Formato
Unknown
|
1.48 MB | Unknown |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
