Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills

Ori Yoran, Alon Talmor, Jonathan Berant

פרסום מחקרי: פרק בספר / בדוח / בכנספרסום בספר כנסביקורת עמיתים

תקציר

Models pre-trained with a language modeling objective possess ample world knowledge and language skills, but are known to struggle in tasks that require reasoning. In this work, we propose to leverage semi-structured tables, and automatically generate at scale question-paragraph pairs, where answering the question requires reasoning over multiple facts in the paragraph. We add a pre-training step over this synthetic data, which includes examples that require 16 different reasoning skills such as number comparison, conjunction, and fact composition. To improve data efficiency, we sample examples from reasoning skills where the model currently errs. We evaluate our approach on three reasoning-focused reading comprehension datasets, and show that our model, PReasM, substantially outperforms T5, a popular pre-trained encoder-decoder model. Moreover, sampling examples based on model errors leads to faster training and higher performance.

שפה מקוריתאנגלית
כותר פרסום המארחACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)
עורכיםSmaranda Muresan, Preslav Nakov, Aline Villavicencio
מוציא לאורAssociation for Computational Linguistics (ACL)
עמודים6016-6031
מספר עמודים16
מסת"ב (אלקטרוני)9781955917216
סטטוס פרסוםפורסם - 2022
אירוע60th Annual Meeting of the Association for Computational Linguistics, ACL 2022 - Dublin, אירלנד
משך הזמן: 22 מאי 202227 מאי 2022
https://aclanthology.org/2022.acl-long.0/

סדרות פרסומים

שםProceedings of the Annual Meeting of the Association for Computational Linguistics
כרך1

כנס

כנס60th Annual Meeting of the Association for Computational Linguistics, ACL 2022
מדינה/אזוראירלנד
עירDublin
תקופה22/05/2227/05/22
כתובת אינטרנט

ASJC Scopus subject areas

  • ???subjectarea.asjc.1700.1706???
  • ???subjectarea.asjc.3300.3310???
  • ???subjectarea.asjc.1200.1203???

טביעת אצבע

להלן מוצגים תחומי המחקר של הפרסום 'Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills'. יחד הם יוצרים טביעת אצבע ייחודית.

פורמט ציטוט ביבליוגרפי