Evaluation has a crucial role in (IR) since it allows for identifying possible points of failure of an IR approach, thus addressing them to improve its effectiveness. Developing tools to support researchers and analysts when analyzing results and investigating strategies to improve IR system performance can help make the analysis easier and more effective. In this paper we discuss a VA-based approach to support the analyst when deciding whether or not to investigate re-ranking to improve the system effectiveness measured after a retrieval run. Our approach is based on effectiveness measures that exploit graded relevance judgements and it provides both a principled and intuitive way to support analysis. A prototype is described and exploited to discuss some case studies based on TREC data. © 2011 Springer-Verlag.
To re-rank or to re-query: Can visual analytics solve this dilemma? / E., Di Buccio; Marco, Dussin; Nicola, Ferro; Ivano, Masiero; Santucci, Giuseppe; Giuseppe, Tino. - STAMPA. - 6941 LNCS:(2011), pp. 119-130. (Intervento presentato al convegno 2nd International Conference on Cross-Language Evaluation Forum, CLEF 2011 tenutosi a Amsterdam nel 19 September 2011 through 22 September 2011) [10.1007/978-3-642-23708-9_14].
To re-rank or to re-query: Can visual analytics solve this dilemma?
SANTUCCI, Giuseppe;
2011
Abstract
Evaluation has a crucial role in (IR) since it allows for identifying possible points of failure of an IR approach, thus addressing them to improve its effectiveness. Developing tools to support researchers and analysts when analyzing results and investigating strategies to improve IR system performance can help make the analysis easier and more effective. In this paper we discuss a VA-based approach to support the analyst when deciding whether or not to investigate re-ranking to improve the system effectiveness measured after a retrieval run. Our approach is based on effectiveness measures that exploit graded relevance judgements and it provides both a principled and intuitive way to support analysis. A prototype is described and exploited to discuss some case studies based on TREC data. © 2011 Springer-Verlag.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
