Punctuation Restoration for Singaporean Spoken Languages: English, Malay, and Mandarin (2022-11-07T00:00:00.000000Z)

TL;DR

This work adopts a slot-filling approach that predicts the presence and type of punctuation marks at each word boundary, similar to the Masked-Language Model approach employed during the pre-training stages of BERT but instead of predicting the masked word, the model predicts masked punctuation.

Abstract

This paper presents the work of restoring punctuation for ASR transcripts generated by multilingual ASR systems. The focus languages are English, Mandarin, and Malay which are three of the most popular languages in Singapore. To the best of our knowledge, this is the first system that can tackle punctuation restoration for these three languages simultaneously. Traditional approaches usually treat the task as a sequential labeling task, however, this work adopts a slot-filling approach that predicts the presence and type of punctuation marks at each word boundary. The approach is similar to the Masked-Language Model approach employed during the pre-training stages of BERT, but instead of predicting the masked word, our model predicts masked punctuation. Additionally, we find that using Jieba11https://github.com/fxsjy/jieba instead of only using the built-in SentencePiece tokenizer of XLM-R can significantly improve the performance of punctuating Mandarin transcripts. Experimental results on English and Mandarin IWSLT2022 datasets and Malay News show that the proposed approach achieved state-of-the-art results for Mandarin with 73.8% F1-score while maintaining a reasonable F1-score for English and Malay, i.e. 74.7% and 78% respectively. Our source code that allows reproducing the results and building a simple web-based application for demonstration purposes is available on Github22https://github.com/AetherPrior/Multilingual-Sentence-Boundary-detection.

Authors

Abhinav Rao

1 papers

Ho Thi-Nga

1 papers

Chng Eng Siong

5 papers

TL;DR

Abstract

Authors

References47 items

Unified Multimodal Punctuation Restoration Framework for Mixed-Modality Corpus

Towards better subtitles: A multilingual approach for punctuation restoration of speech transcripts

Discriminative Self-training for Punctuation Prediction

Automatic punctuation restoration with BERT models

How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models

Word-level BERT-CNN-RNN Model for Chinese Punctuation Restoration

End to End ASR System with Automatic Punctuation Insertion

A 43 Language Multilingual Punctuation Prediction Neural Network Model

The Tatoeba Translation Challenge – Realistic Data Sets for Low Resource and Multilingual MT

Inserting Punctuation to ASR Output in a Real-Time Production Environment

Adversarial Transfer Learning for Punctuation Restoration

Unsupervised Cross-lingual Representation Learning at Scale

Transfer Learning for Punctuation Prediction

On the Cross-lingual Transferability of Monolingual Representations

On the Variance of the Adaptive Learning Rate and Beyond

RoBERTa: A Robustly Optimized BERT Pretraining Approach

How Multilingual is Multilingual BERT?

Cross-lingual Language Model Pretraining

XNLI: Evaluating Cross-lingual Sentence Representations

Self-Attention Based Network for Punctuation Restoration

Attention is All you Need

Bidirectional Recurrent Neural Network with Attention Mechanism for Punctuation Restoration

Punctuation Prediction for Unsegmented Transcript Based on Word Vector

Punctuation: Making a Point in Unsupervised Dependency Parsing

Better Punctuation Prediction with Dynamic Conditional Random Fields

Enriching speech recognition with automatic detection of sentence boundaries and disfluencies

Measuring the readability of automatic speech-to-text transcripts

I and i

Speech recognition with automatic punctuation

A and V

PunKtuator: A Multilingual Punctuation Restoration System for Spoken and Written Text

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations

Punctuation Restoration using Transformer Models for High-and Low-Resource Languages

aTransfer learning for punctuation prediction,o in 2019 Asia-Pacific Signal and Information Processing

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Malay-dataset,” https://github.com/huseinzol05/ malay-dataset, 2018

Deep Learning for Punctuation Restoration in Medical Reports

LSTM for punctuation restoration in speech transcripts

AlumÈ ae, aLstm for punctuation restoration in speech transcripts,o in Sixteenth annual conference of the international speech communication association

aJieba

“Jieba, word tokenizer for chinese,”

I and J

2012 International Workshop on Spoken Language Translation, IWSLT 2012, Hong Kong, December 6-7, 2012

Modeling punctuation prediction as machine translation

Recovering punctuation marks for automatic speech recognition

Γ and B

Punctuation annotation using statistical prosody models.

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names