While contextualized embeddings have produced performance breakthroughs in many Natural Language Processing (NLP) tasks, Word Sense Disambiguation (WSD) has not benefited from them yet. In this paper, we introduce QBERT, a Transformer based architecture for contextualized embeddings which makes use of a coattentive layer to produce more deeply bidirectional representations, better-fitting for the WSD task. As a result, we are able to train a WSD system that beats the state of the art on the concatenation of all evaluation datasets by over 3 points, also outperforming a comparable model using ELMo

Quasi bidirectional encoder representations from transformers for Word Sense Disambiguation / Bevilacqua, Michele; Navigli, Roberto. - In: INTERNATIONAL CONFERENCE RECENT ADVANCES IN NATURAL LANGUAGE PROCESSING. - ISSN 1313-8502. - (2019), pp. 122-131. ((Intervento presentato al convegno Recent Advances in Natural Language Processing tenutosi a Varna; Bulgaria [10.26615/978-954-452-056-4_015].

Quasi bidirectional encoder representations from transformers for Word Sense Disambiguation

Michele Bevilacqua;Roberto Navigli
2019

Abstract

While contextualized embeddings have produced performance breakthroughs in many Natural Language Processing (NLP) tasks, Word Sense Disambiguation (WSD) has not benefited from them yet. In this paper, we introduce QBERT, a Transformer based architecture for contextualized embeddings which makes use of a coattentive layer to produce more deeply bidirectional representations, better-fitting for the WSD task. As a result, we are able to train a WSD system that beats the state of the art on the concatenation of all evaluation datasets by over 3 points, also outperforming a comparable model using ELMo
978-954-452-055-7
File allegati a questo prodotto
File Dimensione Formato  
Bevilacqua_postprint_Quasi_2019.pdf

accesso aperto

Note: DOI 10.26615/978-954-452-056-4_015
Tipologia: Documento in Post-print (versione successiva alla peer review e accettata per la pubblicazione)
Licenza: Creative commons
Dimensione 468.75 kB
Formato Adobe PDF
468.75 kB Adobe PDF Visualizza/Apri PDF
Bevilacqua_Quasi_2019.pdf

accesso aperto

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Creative commons
Dimensione 489.27 kB
Formato Adobe PDF
489.27 kB Adobe PDF Visualizza/Apri PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11573/1350045
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 5
  • ???jsp.display-item.citation.isi??? ND
social impact