Many Hands Make Light Work: Using Essay Traits to Automatically Score Essays (2021-02-01T00:00:00.000000Z)

TL;DR

A way to score essays using a multi-task learning (MTL) approach, where scoring the essay holistically is the primary task, andscoring the essay traits is the auxiliary task, is described.

Abstract

Most research in the area of automatic essay grading (AEG) is geared towards scoring the essay holistically while there has also been little work done on scoring individual essay traits. In this paper, we describe a way to score essays using a multi-task learning (MTL) approach, where scoring the essay holistically is the primary task, and scoring the essay traits is the auxiliary task. We compare our results with a single-task learning (STL) approach, using both LSTMs and BiLSTMs. To find out which traits work best for different types of essays, we conduct ablation tests for each of the essay traits. We also report the runtime and number of training parameters for each system. We find that MTL-based BiLSTM system gives the best results for scoring the essay holistically, as well as performing well on scoring the essay traits. The MTL systems also give a speed-up of between 2.30 to 3.70 times the speed of the STL system, when it comes to scoring the essay and all the traits.

Authors

S. Saha

6 papers

P. Bhattacharyya

12 papers

Rahul Kumar

1 papers

TL;DR

Abstract

Authors

References46 items

Automated Cross-prompt Scoring of Essay Traits

Are pre-trained text representations useful for multilingual and multi-dimensional language proficiency modeling?

Neural Automated Essay Scoring Incorporating Handcrafted Features

Domain-Adaptive Neural Automated Essay Scoring

Can Neural Networks Automatically Score Essay Traits?

Multiple Instance Learning for Content Feedback Localization without Annotation

Should You Fine-Tune BERT for Automated Essay Scoring?

Hierarchical Multi-task Learning for Organization Evaluation of Argumentative Student Essays

Automated Topical Component Extraction Using Neural Network Attention Scores from Source-based Essay Scoring

Happy Are Those Who Grade without Seeing: A Multi-Task Learning Approach to Grade Essays Using Gaze Behaviour

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Give Me More Feedback II: Annotating Thesis Strength and Related Attributes in Student Essays

Eyes are the Windows to the Soul: Predicting the Rating of Text Quality Using Gaze Behaviour

Thank “Goodness”! A Way to Measure Style in Student Essays

Co-Attention Based Neural Network for Source-Dependent Essay Scoring

ASAP++: Enriching the ASAP Automated Essay Grading Dataset with Essay Attribute Scores

Automated essay scoring with string kernels and word embeddings

Towards Evaluating Narrative Quality In Student Writing

SkipFlow: Incorporating Neural Coherence Features for End-to-End Automatic Text Scoring

Attention-based Recurrent Convolutional Neural Network for Automatic Essay Scoring

A Neural Approach to Automated Essay Scoring

Automatic Features for Essay Scoring – An Empirical Study

Modeling Stance in Student Essays

Flexible Domain Adaptation for Automated Essay Scoring Using Correlated Linear Regression

Modeling Argument Strength in Student Essays

Reducing Annotation Efforts in Supervised Short Answer Scoring

Task-Independent Features for Automated Essay Grading

RMSProp and equilibrated adaptive learning rates for non-convex optimization

GloVe: Global Vectors for Word Representation

Lexical Chaining for Measuring Discourse Coherence Quality in Test-taker Essays

Natural Language Processing (Almost) from Scratch

Modeling Organization in Student Essays

Predicting the Fluency of Text with Shallow Structural Features: Case Studies of Machine Translation and Human-Written Text

Long Short-Term Memory

Multitask Learning

The Imminence of Grading Essays by Computer-25 Years Later

Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit.

NLP for Building Educational Applications, 399–409

Enhancing automated essay scoring performance via fine-tuning pre-trained language models with combination of regression and ranking

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Handbook of Automated Essay Evaluation Current Applications and New Directions

NATIONAL UNIVERSITY OF SINGAPORE School of Computing PH.D DEFENCE - PUBLIC SEMINAR Title: Robust Trait-Specific Essay Scoring Using Neural Networks and Density Estimators

Modeling Prompt Adherence in Student Essays

Modeling Thesis Clarity in Student Essays

Annual Meeting of the Association for Computational Lin-guistics , 3994–4004

Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)

Field of Study

Venue Information

Name

Type

URL

Alternate Names