Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills

Ori Yoran, Alon Talmor, Jonathan Berant

نتاج البحث: فصل من :كتاب / تقرير / مؤتمرمنشور من مؤتمرمراجعة النظراء

ملخص

Models pre-trained with a language modeling objective possess ample world knowledge and language skills, but are known to struggle in tasks that require reasoning. In this work, we propose to leverage semi-structured tables, and automatically generate at scale question-paragraph pairs, where answering the question requires reasoning over multiple facts in the paragraph. We add a pre-training step over this synthetic data, which includes examples that require 16 different reasoning skills such as number comparison, conjunction, and fact composition. To improve data efficiency, we sample examples from reasoning skills where the model currently errs. We evaluate our approach on three reasoning-focused reading comprehension datasets, and show that our model, PReasM, substantially outperforms T5, a popular pre-trained encoder-decoder model. Moreover, sampling examples based on model errors leads to faster training and higher performance.

اللغة الأصليةالإنجليزيّة
عنوان منشور المضيفACL 2022 - 60th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)
المحررونSmaranda Muresan, Preslav Nakov, Aline Villavicencio
ناشرAssociation for Computational Linguistics (ACL)
الصفحات6016-6031
عدد الصفحات16
رقم المعيار الدولي للكتب (الإلكتروني)9781955917216
حالة النشرنُشِر - 2022
الحدث60th Annual Meeting of the Association for Computational Linguistics, ACL 2022 - Dublin, إيرلندا
المدة: ٢٢ مايو ٢٠٢٢٢٧ مايو ٢٠٢٢
https://aclanthology.org/2022.acl-long.0/

سلسلة المنشورات

الاسمProceedings of the Annual Meeting of the Association for Computational Linguistics
مستوى الصوت1

!!Conference

!!Conference60th Annual Meeting of the Association for Computational Linguistics, ACL 2022
الدولة/الإقليمإيرلندا
المدينةDublin
المدة٢٢/٠٥/٢٢٢٧/٠٥/٢٢
عنوان الإنترنت

All Science Journal Classification (ASJC) codes

  • !!Computer Science Applications
  • !!Linguistics and Language
  • !!Language and Linguistics

بصمة

أدرس بدقة موضوعات البحث “Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills'. فهما يشكلان معًا بصمة فريدة.

قم بذكر هذا