EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding

Published in

Automatic Speech Recognition & Understanding(2015)

External Links:

Generate Graph DownloadPDF

TL;DR

This paper presents the Eesen framework which drastically simplifies the existing pipeline to build state-of-the-art ASR systems and achieves comparable word error rates (WERs), while at the same time speeding up decoding significantly.

Abstract

The performance of automatic speech recognition (ASR) has improved tremendously due to the application of deep neural networks (DNNs). Despite this progress, building a new ASR system remains a challenging task, requiring various resources, multiple training stages and significant expertise. This paper presents our Eesen framework which drastically simplifies the existing pipeline to build state-of-the-art ASR systems. Acoustic modeling in Eesen involves learning a single recurrent neural network (RNN) predicting context-independent targets (phonemes or characters). To remove the need for pre-generated frame labels, we adopt the connectionist temporal classification (CTC) objective function to infer the alignments between speech and label sequences. A distinctive feature of Eesen is a generalized decoding approach based on weighted finite-state transducers (WFSTs), which enables the efficient incorporation of lexicons and language models into CTC decoding. Experiments show that compared with the standard hybrid DNN systems, Eesen achieves comparable word error rates (WERs), while at the same time speeding up decoding significantly.

Authors

Florian Metze

5 papers

Yajie Miao

1 papers

M. Gowayyed

1 papers

References45 items

Speaker Adaptive Training of Deep Neural Network Acoustic Models Using I-Vectors

Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks

End-to-end attention-based large vocabulary speech recognition

Learning acoustic frame labeling for speech recognition with recurrent neural networks

Deep Convolutional Neural Networks for Large-scale Speech Tasks

EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding

Published in

Automatic Speech Recognition & Understanding(2015)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

Florian Metze

5 papers

Yajie Miao

1 papers

M. Gowayyed

1 papers

References45 items

Speaker Adaptive Training of Deep Neural Network Acoustic Models Using I-Vectors

Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks

End-to-end attention-based large vocabulary speech recognition

Learning acoustic frame labeling for speech recognition with recurrent neural networks

Deep Convolutional Neural Networks for Large-scale Speech Tasks

Deep Speech: Scaling up end-to-end speech recognition

End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results

Improvements to speaker adaptive training of deep neural networks

First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNs

Towards End-To-End Speech Recognition with Recurrent Neural Networks

GMM-free DNN acoustic model training

Hybrid speech recognition with Deep Bidirectional LSTM

Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

Speaker adaptation of context dependent deep neural networks

Speech recognition with deep recurrent neural networks

Adaptation of context-dependent deep neural networks for automatic speech recognition

Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition

Feature engineering in Context-Dependent Deep Neural Networks for conversational speech transcription

OpenFst: A General and Efficient Weighted Finite-State Transducer Library

A Fast Learning Algorithm for Deep Belief Nets

Learning Precise Timing with LSTM Recurrent Networks

A one-pass decoder based on polymorphic linguistic context assignment

Long Short-Term Memory

Learning long-term dependencies with gradient descent is difficult

A novel objective function for improved phoneme recognition using time delay neural networks

Phoneme recognition using time-delay neural networks

A tutorial on hidden Markov models and selected applications in speech recognition

Top Downloads in IEEE Xplore [Reader's Choice]

On speaker adaptation of long short-term memory recurrent neural networks

Lexicon-Free Conversational Speech Recognition with Neural Networks

Towards end-to-end speech recognition for Chinese Mandarin using long short-term memory recurrent neural networks

Listen, attend and spell

Towards speaker adaptive training of deep neural network acoustic models

Long short-term memory recurrent neural network architectures for large scale acoustic modeling

GMM-Free DNN Training

Asynchronous, online, GMM-free training of a context dependent acoustic model for speech recognition

Distributed learning of multilingual DNN feature extractors using GPUs

Improving language-universal feature extraction with deep maxout and convolutional neural networks

Recurrent Neural Networks for Noise Reduction in Robust ASR

The Kaldi Speech Recognition Toolkit

Weighted finite-state transducers in speech recognition

Weighted finite state transducers in speech recognition

Connectionist Temporal Classiﬁcation: Labelling Unsegmented Sequence Data with Recurrent Neural Networks

Connectionist Temporal Classiﬁcation: Labelling Unsegmented Sequences with Recurrent Neural Networks

Field of Study

Computer Science

Journal Information

Name

2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU)

Page

275-282

Venue Information

Name

Automatic Speech Recognition & Understanding

Type

conference

URL

N/A

Alternate Names

IEEE Automatic Speech Recognition and Understanding Workshop
Autom Speech Recognit Underst
ASRU
IEEE Autom Speech Recognit Underst Workshop