Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching

Alissa Ostapenko; Shuly Wintner; Melinda Fricke; Yulia Tsvetkov

doi:10.18653/v1/2022.acl-long.267

Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching

Alissa Ostapenko, Shuly Wintner, Melinda Fricke, Yulia Tsvetkov

Department of Computer Science

University of Haifa

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Natural language processing (NLP) models trained on people-generated data can be unreliable because, without any constraints, they can learn from spurious correlations that are not relevant to the task. We hypothesize that enriching models with speaker information in a controlled, educated way can guide them to pick up on relevant inductive biases. For the speaker-driven task of predicting code-switching points in English-Spanish bilingual dialogues, we show that adding sociolinguistically-grounded speaker features as prepended prompts significantly improves accuracy. We find that by adding influential phrases to the input, speaker-informed models learn useful and explainable linguistic information. To our knowledge, we are the first to incorporate speaker characteristics in a neural model for code-switching, and more generally, take a step towards developing transparent, personalized models that use speaker information in a controlled way.

Original language	American English
Title of host publication	ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)
Editors	Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Publisher	Association for Computational Linguistics (ACL)
Pages	3853-3867
Number of pages	15
ISBN (Electronic)	9781955917216
DOIs	https://doi.org/10.18653/v1/2022.acl-long.267
State	Published - 2022
Event	60th Annual Meeting of the Association for Computational Linguistics, ACL 2022 - Dublin, Ireland Duration: 22 May 2022 → 27 May 2022 https://aclanthology.org/2022.acl-long.0/

Publication series

Name	Proceedings of the Annual Meeting of the Association for Computational Linguistics
Volume	1

Conference

Conference	60th Annual Meeting of the Association for Computational Linguistics, ACL 2022
Country/Territory	Ireland
City	Dublin
Period	22/05/22 → 27/05/22
Internet address	https://aclanthology.org/2022.acl-long.0/

All Science Journal Classification (ASJC) codes

Computer Science Applications
Linguistics and Language
Language and Linguistics

Access to Document

10.18653/v1/2022.acl-long.267

Cite this

Ostapenko, A., Wintner, S., Fricke, M., & Tsvetkov, Y. (2022). Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching. In S. Muresan, P. Nakov, & A. Villavicencio (Eds.), ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers) (pp. 3853-3867). (Proceedings of the Annual Meeting of the Association for Computational Linguistics; Vol. 1). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2022.acl-long.267

Ostapenko, Alissa ; Wintner, Shuly ; Fricke, Melinda et al. / Speaker Information Can Guide Models to Better Inductive Biases : A Case Study On Predicting Code-Switching. ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers). editor / Smaranda Muresan ; Preslav Nakov ; Aline Villavicencio. Association for Computational Linguistics (ACL), 2022. pp. 3853-3867 (Proceedings of the Annual Meeting of the Association for Computational Linguistics).

@inproceedings{d6d13cd9e2f5482b8530c02040669fec,

title = "Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching",

abstract = "Natural language processing (NLP) models trained on people-generated data can be unreliable because, without any constraints, they can learn from spurious correlations that are not relevant to the task. We hypothesize that enriching models with speaker information in a controlled, educated way can guide them to pick up on relevant inductive biases. For the speaker-driven task of predicting code-switching points in English-Spanish bilingual dialogues, we show that adding sociolinguistically-grounded speaker features as prepended prompts significantly improves accuracy. We find that by adding influential phrases to the input, speaker-informed models learn useful and explainable linguistic information. To our knowledge, we are the first to incorporate speaker characteristics in a neural model for code-switching, and more generally, take a step towards developing transparent, personalized models that use speaker information in a controlled way.",

author = "Alissa Ostapenko and Shuly Wintner and Melinda Fricke and Yulia Tsvetkov",

note = "Publisher Copyright: {\textcopyright} 2022 Association for Computational Linguistics.; 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022 ; Conference date: 22-05-2022 Through 27-05-2022",

year = "2022",

doi = "10.18653/v1/2022.acl-long.267",

language = "American English",

series = "Proceedings of the Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics (ACL)",

pages = "3853--3867",

editor = "Smaranda Muresan and Preslav Nakov and Aline Villavicencio",

booktitle = "ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)",

address = "United States",

url = "https://aclanthology.org/2022.acl-long.0/",

}

Ostapenko, A, Wintner, S, Fricke, M & Tsvetkov, Y 2022, Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching. in S Muresan, P Nakov & A Villavicencio (eds), ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers). Proceedings of the Annual Meeting of the Association for Computational Linguistics, vol. 1, Association for Computational Linguistics (ACL), pp. 3853-3867, 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022, Dublin, Ireland, 22/05/22. https://doi.org/10.18653/v1/2022.acl-long.267

Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching. / Ostapenko, Alissa; Wintner, Shuly; Fricke, Melinda et al.
ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers). ed. / Smaranda Muresan; Preslav Nakov; Aline Villavicencio. Association for Computational Linguistics (ACL), 2022. p. 3853-3867 (Proceedings of the Annual Meeting of the Association for Computational Linguistics; Vol. 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Speaker Information Can Guide Models to Better Inductive Biases

T2 - 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022

AU - Ostapenko, Alissa

AU - Wintner, Shuly

AU - Fricke, Melinda

AU - Tsvetkov, Yulia

PY - 2022

Y1 - 2022

N2 - Natural language processing (NLP) models trained on people-generated data can be unreliable because, without any constraints, they can learn from spurious correlations that are not relevant to the task. We hypothesize that enriching models with speaker information in a controlled, educated way can guide them to pick up on relevant inductive biases. For the speaker-driven task of predicting code-switching points in English-Spanish bilingual dialogues, we show that adding sociolinguistically-grounded speaker features as prepended prompts significantly improves accuracy. We find that by adding influential phrases to the input, speaker-informed models learn useful and explainable linguistic information. To our knowledge, we are the first to incorporate speaker characteristics in a neural model for code-switching, and more generally, take a step towards developing transparent, personalized models that use speaker information in a controlled way.

AB - Natural language processing (NLP) models trained on people-generated data can be unreliable because, without any constraints, they can learn from spurious correlations that are not relevant to the task. We hypothesize that enriching models with speaker information in a controlled, educated way can guide them to pick up on relevant inductive biases. For the speaker-driven task of predicting code-switching points in English-Spanish bilingual dialogues, we show that adding sociolinguistically-grounded speaker features as prepended prompts significantly improves accuracy. We find that by adding influential phrases to the input, speaker-informed models learn useful and explainable linguistic information. To our knowledge, we are the first to incorporate speaker characteristics in a neural model for code-switching, and more generally, take a step towards developing transparent, personalized models that use speaker information in a controlled way.

UR - http://www.scopus.com/inward/record.url?scp=85146160188&partnerID=8YFLogxK

U2 - 10.18653/v1/2022.acl-long.267

DO - 10.18653/v1/2022.acl-long.267

M3 - Conference contribution

T3 - Proceedings of the Annual Meeting of the Association for Computational Linguistics

SP - 3853

EP - 3867

BT - ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)

A2 - Muresan, Smaranda

A2 - Nakov, Preslav

A2 - Villavicencio, Aline

PB - Association for Computational Linguistics (ACL)

Y2 - 22 May 2022 through 27 May 2022

ER -

Ostapenko A, Wintner S, Fricke M, Tsvetkov Y. Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching. In Muresan S, Nakov P, Villavicencio A, editors, ACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers). Association for Computational Linguistics (ACL). 2022. p. 3853-3867. (Proceedings of the Annual Meeting of the Association for Computational Linguistics). doi: 10.18653/v1/2022.acl-long.267

Speaker Information Can Guide Models to Better Inductive Biases: A Case Study On Predicting Code-Switching

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this