Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset
Published in
International Conference on Learning Representa...(2018)External Links:
TL
TL;DR
By using notes as an intermediate representation, a suite of models capable of transcribing, composing, and synthesizing audio waveforms with coherent musical structure on timescales spanning six orders of magnitude are trained, a process the authors call Wave2Midi2Wave.
Abstract
Generating musical audio directly with neural networks is notoriously difficult because it requires coherently modeling structure at many different timescales. Fortunately, most music is also highly structured and can be represented as discrete note events played on musical instruments. Herein, we show that by using notes as an intermediate representation, we can train a suite of models capable of transcribing, composing, and synthesizing audio waveforms with coherent musical structure on timescales spanning six orders of magnitude (~0.1 ms to ~100 s), a process we call Wave2Midi2Wave. This large advance in the state of the art is enabled by our release of the new MAESTRO (MIDI and Audio Edited for Synchronous TRacks and Organization) dataset, composed of over 172 hours of virtuosic piano performances captured with fine alignment (~3 ms) between note labels and audio waveforms. The networks and the dataset together present a promising approach toward creating new expressive and interpretable neural models of music.
References22 items
1
Deep Polyphonic ADSR Piano Note Transcription
2
An Improved Relative Self-Attention Mechanism for Transformer with Application to Music Generation
3
The challenge of realistic music generation: modelling raw audio at scale
4
Conditioning Deep Generative Raw Audio Models for Structured Automatic Music
5