The evaluation of several tasks in lexical semantics is often limited by the lack of large numbers of manual annotations, not only for training purposes, but also for testing purposes. Word Sense Disambiguation (WSD) is a case in point, as hand-labeled data sets are particularly hard and time-consuming to create. Consequently, evaluations tend to be performed on a small scale, which does not allow for in-depth analysis of the factors that determine a system’s performance. In this article we address this issue by means of a realistic simulation of large-scale evaluation for the WSD task. We do this by providing two main contributions: First, we put forward two novel approaches to the wide-coverage generation of semantically aware pseudowords (i.e., artificial words capable of modeling real polysemous words); second, we leverage the most suitable type of pseudoword to create large pseudosense-annotated corpora, which enable a large-scale experimental framework for the comparison of st
A Large-scale Pseudoword-based Evaluation Framework for State-of-the-Art Word Sense Disambiguation / Pilehvar, MOHAMMED TAHER; Navigli, Roberto. - In: COMPUTATIONAL LINGUISTICS. - ISSN 1530-9312. - ELETTRONICO. - 4:40(2014), pp. 837-881. [10.1162/COLI_a_00202]
A Large-scale Pseudoword-based Evaluation Framework for State-of-the-Art Word Sense Disambiguation
PILEHVAR, MOHAMMED TAHER;NAVIGLI, ROBERTO
2014
Abstract
The evaluation of several tasks in lexical semantics is often limited by the lack of large numbers of manual annotations, not only for training purposes, but also for testing purposes. Word Sense Disambiguation (WSD) is a case in point, as hand-labeled data sets are particularly hard and time-consuming to create. Consequently, evaluations tend to be performed on a small scale, which does not allow for in-depth analysis of the factors that determine a system’s performance. In this article we address this issue by means of a realistic simulation of large-scale evaluation for the WSD task. We do this by providing two main contributions: First, we put forward two novel approaches to the wide-coverage generation of semantically aware pseudowords (i.e., artificial words capable of modeling real polysemous words); second, we leverage the most suitable type of pseudoword to create large pseudosense-annotated corpora, which enable a large-scale experimental framework for the comparison of stI documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.