TY - GEN
T1 - Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss
AU - Plank, Barbara
AU - Søgaard, Anders
AU - Goldberg, Yoav
N1 - Publisher Copyright: © 2016 Association for Computational Linguistics.
PY - 2016
Y1 - 2016
N2 - Bidirectional long short-term memory (bi-LSTM) networks have recently proven successful for various NLP sequence modeling tasks, but little is known about their reliance to input representations, target languages, data set size, and label noise. We address these issues and evaluate bi-LSTMs with word, character, and unicode byte embeddings for POS tagging. We compare bi-LSTMs to traditional POS taggers across languages and data sizes. We also present a novel bi-LSTM model, which combines the POS tagging loss function with an auxiliary loss function that accounts for rare words. The model obtains state-of-the-art performance across 22 languages, and works especially well for morphologically complex languages. Our analysis suggests that bi-LSTMs are less sensitive to training data size and label corruptions (at small noise levels) than previously assumed.
AB - Bidirectional long short-term memory (bi-LSTM) networks have recently proven successful for various NLP sequence modeling tasks, but little is known about their reliance to input representations, target languages, data set size, and label noise. We address these issues and evaluate bi-LSTMs with word, character, and unicode byte embeddings for POS tagging. We compare bi-LSTMs to traditional POS taggers across languages and data sizes. We also present a novel bi-LSTM model, which combines the POS tagging loss function with an auxiliary loss function that accounts for rare words. The model obtains state-of-the-art performance across 22 languages, and works especially well for morphologically complex languages. Our analysis suggests that bi-LSTMs are less sensitive to training data size and label corruptions (at small noise levels) than previously assumed.
UR - http://www.scopus.com/inward/record.url?scp=85009255621&partnerID=8YFLogxK
U2 - https://doi.org/10.18653/v1/p16-2067
DO - https://doi.org/10.18653/v1/p16-2067
M3 - منشور من مؤتمر
T3 - 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Short Papers
SP - 412
EP - 418
BT - 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016 - Short Papers
PB - Association for Computational Linguistics (ACL)
T2 - 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016
Y2 - 7 August 2016 through 12 August 2016
ER -