Transformer-Based Multi-Aspect Multi-Granularity Non-Native English Speaker Pronunciation Assessment (2022-05-06T00:00:00.000000Z)

TL;DR

This work trains a Goodness Of Pronunciation feature-based Transformer (GOPT) with multi-task learning and shows that GOPT achieves the best results on speechocean762 with a public automatic speech recognition (ASR) acoustic model trained on Librispeech.

Abstract

Automatic pronunciation assessment is an important technology to help self-directed language learners. While pronunciation quality has multiple aspects including accuracy, fluency, completeness, and prosody, previous efforts typically only model one aspect (e.g., accuracy) at one granularity (e.g., at the phoneme-level). In this work, we explore modeling multi-aspect pronunciation assessment at multiple granularities. Specifically, we train a Goodness Of Pronunciation feature-based Transformer (GOPT) with multi-task learning. Experiments show that GOPT achieves the best results on speechocean762 with a public automatic speech recognition (ASR) acoustic model trained on Librispeech.

Authors

Yuan Gong

7 papers

James R. Glass

16 papers

I. Chu

1 papers

TL;DR

Abstract

Authors

References27 items

Deep Feature Transfer Learning for Automatic Pronunciation Assessment

Explore wav2vec 2.0 for Mispronunciation Detection

speechocean762: An Open-Source Non-native English Speech Corpus For Pronunciation Assessment

Automatic Scoring at Multi-Granularity for L2 Pronunciation

Context-aware Goodness of Pronunciation for Computer-Assisted Pronunciation Training

Investigating the role of L1 in automatic pronunciation evaluation of L2 speech

TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation

Attention is All you Need

Intonation classification for L2 English speech using multi-distribution deep neural networks

Librispeech: An ASR corpus based on public domain audio books

Improved mispronunciation detection with deep neural network trained acoustic models and transfer learning based logistic regression classifiers

Improved approaches of modeling and detecting Error Patterns with empirical analysis for Computer-Aided Pronunciation Training

Automatic intonation assessment for computer aided language learning

An overview of spoken language technology for education

Automatic scoring of non-native spontaneous speech in tests of spoken English

Automatic mispronunciation detection for Mandarin

Automatic syllable stress detection using prosodic features for pronunciation evaluation of language learners

Phone-level pronunciation scoring and assessment for interactive language learning

Quantitative assessment of second language learners' fluency by means of automatic speech recognition technology.

Quantitative assessment of second language learners' fluency: an automatic approach

Building a Large Annotated Corpus of English: The Penn Treebank

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

“Automatic error detection in pronunciation training: Where we are and where we need to go

Using non-native error patterns to improve pronunciation verification

Analysis and utilization of MLLR speaker adaptation technique for learners' pronunciation evaluation

Automatic pronunciation scoring of words and sentences independent from the non-native's first language

Automatic prosodic analysis for computer aided pronunciation teaching

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names