First Steps Towards NLP-based Formative Feedback to Improve Scientific Writing in Hebrew

As scientific writing is an important 21st century skill, its development is a major goal in high school science education. Research shows that developing scientific writing skills requires frequent and tailored feedback, which teachers, who face large classes and limited time for personalized instruction, struggle to give. Natural Language Processing (NLP) technologies offer great promise to assist teachers in this process by automating some of the analysis. However, in Hebrew, the use of NLP in computer-supported writing instruction was until recently hindered by the lack of publicly available resources. In this paper, we present initial results from a study that aims to develop NLP-based techniques to assist teachers in providing personalized feedback in scientific writing in Hebrew, which might be applicable to other languages as well. We focus on writing inquiry reports in Biology, and specifically, on the task of automatically identifying whether the report contains a properly defined research question. This serves as a proof-of-concept of whether we can build a pipeline that identifies major components of the report and match them to a predefined grading rubric. To achieve this, we collected several hundreds of reports, annotated them according to a grading rubric to create a supervised data set, and built a machine-learning algorithm that uses NLP-based features. The results show that our model can accurately identify the research question or its absence. To the best of our knowledge, this is the first paper to report on the application of Hebrew NLP for formative assessment in K-12 science education.