STEIN: Speeding up evaluation activities with a seamless testing environment integrator

The evaluation of an information visualization system is a complex activity, involving the understanding of both the visualization itself and the process that it is meant to support. Moreover, if the evaluation activity includes a task based user study, it requires a considerable effort, involving both conceptual (e.g., the definition of user tasks) and technical (e.g., logging of the relevant user actions while using the system) aspects. The solution presented in this paper, STEIN (Seamless Testing Environment INtegrator), allows integrating the system under evaluation with the questions that have been designed for the user study, tracing the user's activities and automatically collecting the user's answers using the events that are generated while interacting with the system. This results in a substantial reduction of the effort associated with technical activities, thus allowing the evaluation designer to focus mainly on the conceptual aspects. A prototype of the system is available for download at awareserver.dis.uniroma1.it:8080/stein.