Adaptive parser-centric text normalization

Congle Zhang, Tyler Baldwin, Howard Ho, Benny Kimelfeld, Yunyao Li

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Text normalization is an important first step towards enabling many Natural Language Processing (NLP) tasks over informal text. While many of these tasks, such as parsing, perform the best over fully grammatically correct text, most existing text normalization approaches narrowly define the task in the word-to-word sense; that is, the task is seen as that of mapping all out-of-vocabulary non-standard words to their in-vocabulary standard forms. In this paper, we take a parser-centric view of normalization that aims to convert raw informal text into grammatically correct text. To understand the real effect of normalization on the parser, we tie normalization performance directly to parser performance. Additionally, we design a customizable framework to address the often overlooked concept of domain adaptability, and illustrate that the system allows for transfer to new domains with a minimal amount of data and effort. Our experimental study over datasets from three domains demonstrates that our approach outperforms not only the state-of-the-art wordto-word normalization techniques, but also manual word-to-word annotations.

Original languageEnglish
Title of host publicationLong Papers
Number of pages10
StatePublished - 2013
Externally publishedYes
Event51st Annual Meeting of the Association for Computational Linguistics, ACL 2013 - Sofia, Bulgaria
Duration: 4 Aug 20139 Aug 2013

Publication series

NameACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference


Conference51st Annual Meeting of the Association for Computational Linguistics, ACL 2013

All Science Journal Classification (ASJC) codes

  • Language and Linguistics
  • Linguistics and Language


Dive into the research topics of 'Adaptive parser-centric text normalization'. Together they form a unique fingerprint.

Cite this