Real-Time Packet Loss Concealment With Mixed Generative and Predictive Model
TL
TL;DR
A hybrid neural PLC architecture where the missing speech is synthesized using a generative model conditioned using a predictive model to achieve natural concealment that surpasses the quality of existing conventional PLC algorithms and ranked second in the Interspeech 2022 PLC Challenge.
Abstract
As deep speech enhancement algorithms have recently demonstrated capabilities greatly surpassing their traditional counterparts for suppressing noise, reverberation and echo, attention is turning to the problem of packet loss concealment (PLC). PLC is a challenging task because it not only involves real-time speech synthesis, but also frequent transitions between the received audio and the synthesized concealment. We propose a hybrid neural PLC architecture where the missing speech is synthesized using a generative model conditioned using a predictive model. The resulting algorithm achieves natural concealment that surpasses the quality of existing conventional PLC algorithms and ranked second in the Interspeech 2022 PLC Challenge. We show that our solution not only works for uncompressed audio, but is also applicable to a modern speech codec.
References36 items
1
INTERSPEECH 2022 Audio Deep Packet Loss Concealment Challenge
2
Neural Speech Synthesis on a Shoestring: Improving the Efficiency of Lpcnet
3
Dnsmos P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors
4
Adversarial Auto-Encoding for Packet Loss Concealment
5