![]() Bridge languages are languages for which data is mined across groups. Language families group languages that are similar and all languages within a group are mined against all other languages in that group. To address the prohibitiveness of mining data for each and every pair of languages, the authors demonstrate a method of sparse mining by defining language family groupings and bridge languages. CCAligned first pre-selects documents that are likely to contain mutual translations, then mines for bitexts within the paired documents.īoth approaches perform sentence comparisons using language agnostic semantic embeddings generated by the LASER encoder. CCMatrix uses a global approach in mining for bitexts - it compares each unique sentence in one language to all unique sentences in another language to find bitext pairs. The experimenters leverage and extend two multilingual bitext corpi, CCMatrix and CCAligned. This type of data makes up the training and test sets for a Transformer based translation model. Additionally, languages were restricted to those for which public evaluation data and monolingual data existed.Ī definition : pairs of sentences that are translations of one another are called bitext data. Language coverage was chosen to include widely spoken languages from geographically diverse language families and a diversity of scripts, with the objective of high coverage of worldwide languages. In order to train such a large model the authors implemented recent dense scaling advancements such as optimizer state sharding and gradient checkpointing for reducing the memory required to process states, and model parallelism to split training across multiple devices. Lastly, they show how utilizing backtranslated synthetic bitexts can improve translations in otherwise low resource languages. They define a mining strategy based on groupings of language families and languages chosen to span across the groupings dubbed bridge languages, and demonstrate how this strategy can be used to improve sparse mining over the language pair matrix vs random sampling. The authors leverage and extend the multilingual translation corpi CCMatrix and CCAligned for training and test data. The training of such a model was possible due to improvements in the mining of multilingual translation data. Additionally human translators rated M2M higher than English-Centric models in a blind test. M2M_100 outperforms English-Centric multilingual models, published bilingual models, and other published direct translation multilingual models. ![]() ![]() Doing away with the need to use English as an intermediary language as is common in English-Centric translation models. Able to directly translate between any pair of 100 languages. This paper introduces multilingual translation model, M2M_100, that performs direct translation between languages. Then, a description of the datasets produced in order to train the model, some strategies employed to train such a large model, and finally a breakdown of the M2M_100 model design. The following write-up will start with a synopsis to present the overall summary of the paper. In addition to the headline achievement, two additional advancements were made through the research presented in the paper - methods for multilingual data mining culminating in expansion of two large multilingual translation corpora, and implementation of methods for improving the training of very large deep learning models to the context of translation models. The headline advancement from this paper is the M2M_100 pre-trained multilingual translation model. This posts aims to summarize the findings presented in the paper entitled, “ Beyond English-Centric Multilingual Machine Translation”.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |