NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Rates
TL
TL;DR
NU-Wave 2 is introduced, a diffusion model for neural audio upsampling that enables the generation of 48 kHz audio signals from inputs of various sampling rates with a single model and requires fewer parameters than other models.
Abstract
Conventionally, audio super-resolution models fixed the initial and the target sampling rates, which necessitate the model to be trained for each pair of sampling rates. We introduce NU-Wave 2, a diffusion model for neural audio upsampling that enables the generation of 48 kHz audio signals from inputs of various sampling rates with a single model. Based on the architecture of NU-Wave, NU-Wave 2 uses short-time Fourier convolution (STFC) to generate harmonics to resolve the main failure modes of NU-Wave, and incorporates bandwidth spectral feature transform (BSFT) to condition the bandwidths of inputs in the frequency domain. We experimentally demonstrate that NU-Wave 2 produces high-resolution audio regardless of the sampling rate of input while requiring fewer parameters than other models. The official code and the audio samples are available at https://mindslab-ai.github.io/nuwave2.
References30 items
1
Classifier-Free Diffusion Guidance
2
Tunet: A Block-Online Bandwidth Extension Model Based On Transformers And Self-Supervised Pretraining
3
VoiceFixer: Toward General Speech Restoration With Neural Vocoder
4
Resolution-robust Large Mask Inpainting with Fourier Convolutions
5