natural-language-processing-5

Speech-to-Text Translation

3260 papers • 126 benchmarks • 313 datasets

Translate audio signals of speech in one language into text in a foreign language, either in an end-to-end or cascade manner.

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in speech-to-text-translation-5

Trend

Dataset

Best Model

Actions

MuST-C EN->DE

MuST-C EN->ES

MuST-C EN->FR

Libraries

i

Use these libraries to find speech-to-text-translation-5 models and implementations

formiel/fairseq

2 papers 17

Datasets

MuST-C

CoVoST

Kosp2e

Subtasks

Simultaneous Speech-to-Text Translation

Most implemented papers

Fairseq S2T: Fast Speech-to-Text Modeling with Fairseq

Changhan Wang, J. Pino, Anne Wu, Dmytro Okhonko, Yun Tang, Xutai Ma•Sat Oct 10 2020

State-of-the-art RNN-based as well as Transformer-based models and open-source detailed training recipes are implemented and seamlessly integrated into S2T workflows for multi-task learning or transfer learning.

318

Content

MuST-C EN->FR

libri-trans

MuST-C

MuST-C EN->NL

0

Paper Graph

CoVoST 2 and Massively Multilingual Speech-to-Text Translation

Changhan Wang, J. Pino, Anne Wu•Sun Jul 19 2020

CoVoST 2 is released, a large-scale multilingual speech translation corpus covering translations from 21 languages into English and from English into 15 languages, which represents the largest open dataset available to date from total volume and language coverage perspective.

158 0

Paper Graph

Learning Shared Semantic Space for Speech-to-Text Translation

Lei Li, Heng Ji, Mingxuan Wang, Chi Han•Thu May 06 2021

By projecting audio and text features to a common semantic representation, Chimera unifies MT and ST tasks and boosts the performance on ST benchmarks, MuST-C and Augmented Librispeech, to a new state-of-the-art.

85 0

Paper Graph

Lightweight Adapter Tuning for Multilingual Speech Translation

Jiatao Gu, Changhan Wang, J. Pino, L. Besacier, Hang Le, D. Schwab•Tue Jun 01 2021

This paper proposes a comprehensive analysis of adapters for multilingual speech translation (ST) and shows that adapters can be used to efficiently specialize ST to specific language pairs with a low extra cost in terms of parameters.

95 0

Paper Graph

SHAS: Approaching optimal Segmentation for End-to-End Speech Translation

M. Costa-jussà, Yiannis (Ioannis) Tsiamas, Gerard I. Gállego, José A. R. Fonollosa•Tue Feb 08 2022

This work proposes Supervised Hybrid Audio Segmentation (SHAS), a method that can effectively learn the optimal segmentation from any manually segmented speech corpus, and which retains 95-98% of the manual segmentation's BLEU score, compared to the 87-93%" of the best existing methods.

52 0

Paper Graph

PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit

Renjie Zheng, Junkun Chen, Xintong Li, Liang Huang, Hui Zhang, Tian Yuan, Yuxin Huang, Xiaojie Chen, Enlei Gong, Zeyu Chen, Xiaoguang Hu, Dianhai Yu, Yanjun Ma•Thu May 19 2022

The design philosophy and core architecture of PaddleSpeech is described to support several essential speech- to-text and text-to-speech tasks to achieve competitive or state-of-the-art performance on various speech datasets.

31 0

Paper Graph

Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluation

L. Besacier, A. Kocabiyikoglu, Olivier Kraif•Thu Feb 08 2018

This paper augments an existing (monolingual) corpus: LibriSpeech with an existing corpus derived from read audiobooks from the LibriVox project, and shows that the automatic alignments scores are reasonably correlated with the human judgments of the bilingual alignment quality.

109 0

Paper Graph

Adding a benchmark result helps the community track progress.