A Neural Pairwise Ranking Model for Readability Assessment (2022-03-14T00:00:00.000000Z)

TL;DR

This paper proposes the first neural pairwise ranking model for ARA, and shows the first results of cross-lingual, zero-shot evaluation of ARA with neural models.

Abstract

Automatic Readability Assessment (ARA), the task of assigning a reading level to a text, is traditionally treated as a classification problem in NLP research. In this paper, we propose the first neural, pairwise ranking approach to ARA and compare it with existing classification, regression, and (non-neural) ranking methods. We establish the performance of our approach by conducting experiments with three English, one French and one Spanish datasets. We demonstrate that our approach performs well in monolingual single/cross corpus testing scenarios and achieves a zero-shot cross-lingual ranking accuracy of over 80% for both French and Spanish when trained on English data. Additionally, we also release a new parallel bilingual readability dataset, that could be useful for future research. To our knowledge, this paper proposes the first neural pairwise ranking model for ARA, and shows the first results of cross-lingual, zero-shot evaluation of ARA with neural models.

Authors

Sowmya Vajjala

4 papers

Justin Lee

1 papers

TL;DR

Abstract

Authors

References66 items

Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic Features

Using Broad Linguistic Complexity Modeling for Cross-Lingual Readability Assessment

Transformers: State-of-the-Art Natural Language Processing

An Analysis of Transfer Learning Methods for Multilingual Readability Assessment

Linguistic Features for Readability Assessment

Learning-to-Rank with BERT in TF-Ranking

ReadNet: A Hierarchical Transformer Framework for Web Article Readability Analysis

Readability Assessment of Internet-based Patient Education Materials Related to Treatment Options for Benign Prostatic Hyperplasia.

Text as Environment: A Deep Reinforcement Learning Text Readability Assessment Model

Controlling Text Complexity in Neural Machine Translation

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Is cross‐lingual readability assessment possible?

Assessment of Use, Specificity, and Readability of Written Clinical Informed Consent Forms for Patients With Cancer Undergoing Radiotherapy.

Supervised and Unsupervised Neural Approaches to Text Readability

SciPy 1.0: fundamental algorithms for scientific computing in Python

Multiattentive Recurrent Neural Network Architecture for Multilingual Readability Assessment

TF-Ranking: Scalable TensorFlow Library for Learning-to-Rank

Learning Groupwise Multivariate Scoring Functions Using Deep Neural Networks

The LambdaLoss Framework for Ranking Metric Optimization

Enriching Word Embeddings with Domain Knowledge for Readability Assessment

Assessing the Readability of Web Search Results for Searchers with Dyslexia

OneStopEnglish corpus: A new corpus for automatic readability assessment and text simplification

Language Modeling by Clustering with Word Embeddings for Text Readability Assessment

Psycholinguistic Models of Sentence Processing Improve Sentence Readability Ranking

Enriching Word Vectors with Subword Information

Text Readability Assessment for Second Language Learners

Hierarchical Attention Networks for Document Classification

Assessing Relative Sentence Complexity using an Incremental CCG Parser

RankDCG: Rank-Ordering Evaluation Measure

Graph-based Coherence Modeling For Assessing Readability

Problems in Current Text Simplification Research: New Data Can Help

GloVe: Global Vectors for Word Representation

Coherence and Cohesion for the Assessment of Text Readability

A Language-Independent Approach to Automatic Text Difficulty Assessment for Second-Language Learners

Measuring Readability in Financial Disclosures

Efficient Estimation of Word Representations in Vector Space

New Readability Measures for Bangla and Hindi Texts

Readability Classification for German using Lexical, Syntactic, and Morphological Features

On Improving the Accuracy of Readability Classification using Insights from Second Language Acquisition

Ranking-based readability assessment for early primary children’s literature

Characterizing web content, user interests, and search behavior by reading level and topic

READ–IT: Assessing Readability of Italian Texts with a View to Text Simplification

Scikit-learn: Machine Learning in Python

Sorting Texts by Readability

Cognitively Motivated Features for Readability Assessment

Unlocking Language: The Classic Readability Studies

Revisiting Readability: A Unified Framework for Predicting Text Quality

An Analysis of Statistical Models and Features for Reading Difficulty Prediction

Combining Lexical and Grammatical Features to Improve Readability Measures for First and Second Language Texts

Automatic Evaluation of Information Ordering: Kendall’s Tau

A statistical model for scientific readability

Measuring Reading Comprehension with the Lexile Framework.

A training algorithm for optimal margin classifiers

Practical Nonparametric Statistics

A new readability yardstick.

Knowledge-Rich BERT Embeddings for Readability Assessment

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Controlling the Reading Level of Machine Translation Output

The textevaluator tool: Helping teachers and test developers select texts for use in instruction and assessment

An “AI readability” Formula for French as a Foreign Language

Common core standards: The new us intended curriculum

A machine learning approach to reading level assessment

A method for measuring the vocabulary burden of textbooks

Demonstration of the use pairwise ranking to achieve cross-corpus compatibility in ARA

A new neural pairwise ranking model with an application to automatic readability assessment

First evidence of zero shot

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL