Analyzing Gender Representation in Multilingual Models

Hila Gonen, Shauli Ravfogel, Yoav Goldberg

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Multilingual language models were shown to allow for nontrivial transfer across scripts and languages. In this work, we study the structure of the internal representations that enable this transfer. We focus on the representation of gender distinctions as a practical case study, and examine the extent to which the gender concept is encoded in shared subspaces across different languages. Our analysis shows that gender representations consist of several prominent components that are shared across languages, alongside language-specific components. The existence of language-independent and language-specific components provides an explanation for an intriguing empirical observation we make: while gender classification transfers well across languages, interventions for gender removal, trained on a single language, do not transfer easily to others.

Original languageEnglish
Title of host publicationACL 2022 - 7th Workshop on Representation Learning for NLP, RepL4NLP 2022 - Proceedings of the Workshop
PublisherAssociation for Computational Linguistics (ACL)
Pages67-77
Number of pages11
ISBN (Electronic)9781955917483
StatePublished - 2022
Event7th Workshop on Representation Learning for NLP, RepL4NLP 2022 at ACL 2022 - Dublin, Ireland
Duration: 26 May 2022 → …

Publication series

NameProceedings of the Annual Meeting of the Association for Computational Linguistics

Conference

Conference7th Workshop on Representation Learning for NLP, RepL4NLP 2022 at ACL 2022
Country/TerritoryIreland
CityDublin
Period26/05/22 → …

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Linguistics and Language
  • Language and Linguistics

Fingerprint

Dive into the research topics of 'Analyzing Gender Representation in Multilingual Models'. Together they form a unique fingerprint.

Cite this