Visual rank analysis for search engine benchmarking and efficient navigation

In many important applications, the search for non-standard data types is essential. E.g., digital libraries and multimedia database systems offer content-based search functionality for images and 3D documents. Contrary to the annotation-based approach, where information manually attached to the data objects if used for retrieval, in content-based retrieval, automatically derived meta-data is used. However, the quality of the meta data is crucial, and often, it a priori is not clear which meta data is best suited to execute a user-issued query. Owing to the multi-meta data problem, two crucial questions arise: (a) how can different meta data (feature vector) schemas be benchmarked to assess their suitability for solving the retrieval problem effectively, and (b) how to support the user with issuing queries to the retrieval system, considering different choices for the type of meta data to engage in the search. In this paper, we address these questions in a two-fold contribution. Based on the DARE visualization system, we first introduce an approach for the visual benchmarking of multiple meta data formats on a ground truth benchmark, supporting the optimization stage of the multimedia database design. We secondly propose a simple, yet effective visual interface to multiple, long lists (rankings) of answer objects for the user. The latter, based on relevance feedback information supplied by the user, allows the effective identification of the meta data schema best suited for executing the similarity queries at hand