In this paper, we propose an architecture for machine translation (MT) capable of obtaining multilingual sentence representations by incorporating an intermediate attention bridge that is shared across all languages. We train the model with language-specific encoders and decoders that are connected through an inner-attention layer on the encoder side. The attention bridge exploits the semantics from each language for translation and develops into a language-agnostic meaning representation that can efficiently be used for transfer learning. We present a new framework for the efficient development of multilingual neural machine translation (NMT) using this model and scheduled training. We have tested the approach in a systematic way with a multi-parallel data set. The model achieves substantial improvements over strong bilingual models and performs well for zero-shot translation, which demonstrates its ability of abstraction and transfer learning.

Multilingual NMT with a Language-Independent Attention Bridge / Vázquez, Raúl; Raganato, Alessandro; Tiedemann, Jörg; Creutz, Mathias. - (2019), pp. 33-39. (Intervento presentato al convegno The 4th Workshop on Representation Learning for NLP (RepL4NLP-2019) tenutosi a Florence; Italy) [10.18653/v1/W19-4305].

Multilingual NMT with a Language-Independent Attention Bridge

Raganato, Alessandro;
2019

Abstract

In this paper, we propose an architecture for machine translation (MT) capable of obtaining multilingual sentence representations by incorporating an intermediate attention bridge that is shared across all languages. We train the model with language-specific encoders and decoders that are connected through an inner-attention layer on the encoder side. The attention bridge exploits the semantics from each language for translation and develops into a language-agnostic meaning representation that can efficiently be used for transfer learning. We present a new framework for the efficient development of multilingual neural machine translation (NMT) using this model and scheduled training. We have tested the approach in a systematic way with a multi-parallel data set. The model achieves substantial improvements over strong bilingual models and performs well for zero-shot translation, which demonstrates its ability of abstraction and transfer learning.
2019
The 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)
attention bridge; machine translation; sentence representation
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
Multilingual NMT with a Language-Independent Attention Bridge / Vázquez, Raúl; Raganato, Alessandro; Tiedemann, Jörg; Creutz, Mathias. - (2019), pp. 33-39. (Intervento presentato al convegno The 4th Workshop on Representation Learning for NLP (RepL4NLP-2019) tenutosi a Florence; Italy) [10.18653/v1/W19-4305].
File allegati a questo prodotto
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1553739
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 24
  • ???jsp.display-item.citation.isi??? 9
social impact