PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition (2019-12-21T00:00:00.000000Z)

TL;DR

This paper proposes pretrained audio neural networks (PANNs) trained on the large-scale AudioSet dataset, and investigates the performance and computational complexity of PANNs modeled by a variety of convolutional neural networks.

Abstract

Audio pattern recognition is an important research topic in the machine learning area, and includes several tasks such as audio tagging, acoustic scene classification, music classification, speech emotion classification and sound event detection. Recently, neural networks have been applied to tackle audio pattern recognition problems. However, previous systems are built on specific datasets with limited durations. Recently, in computer vision and natural language processing, systems pretrained on large-scale datasets have generalized well to several tasks. However, there is limited research on pretraining systems on large-scale datasets for audio pattern recognition. In this paper, we propose pretrained audio neural networks (PANNs) trained on the large-scale AudioSet dataset. These PANNs are transferred to other audio related tasks. We investigate the performance and computational complexity of PANNs modeled by a variety of convolutional neural networks. We propose an architecture called Wavegram-Logmel-CNN using both log-mel spectrogram and waveform as input feature. Our best PANN system achieves a state-of-the-art mean average precision (mAP) of 0.439 on AudioSet tagging, outperforming the best previous system of 0.392. We transfer PANNs to six audio pattern recognition tasks, and demonstrate state-of-the-art performance in several of those tasks. We have released the source code and pretrained models of PANNs: https://github.com/qiuqiangkong/audioset_tagging_cnn.

Authors

Mark D. Plumbley

11 papers

Qiuqiang Kong

11 papers

Yuxuan Wang

8 papers

TL;DR

Abstract

Authors

References62 items

A Deep Residual Network for Large-Scale Acoustic Scene Analysis

musicnn: Pre-trained convolutional neural networks for music audio tagging

Integrating the Data Augmentation Scheme with Various Classifiers for Acoustic Scene Modeling

Classification of snoring sound based on a recurrent neural network

Generalisation in Environmental Sound Classification: The ‘Making Sense of Sounds’ Data Set and Challenge

SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition

Cross-task learning for audio tagging, sound event detection and spatial localization: DCASE 2019 baseline systems

Weakly Labelled AudioSet Tagging With Attention Neural Networks

Bottom-up broadcast neural network for music genre classification

A Comparison of Five Multiple Instance Learning Pooling Functions for Sound Event Detection with Weak Labeling

General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline

A multi-device dataset for urban acoustic scene classification

Learning to Recognize Transient Sound Events using Attentional Supervision

Multi-level attention model for weakly supervised audio classification

Detection and Classification of Acoustic Scenes and Events: Outcome of the DCASE 2016 Challenge

MobileNetV2: Inverted Residuals and Linear Bottlenecks

Spectrogram based multi-task audio classification

DCASE2017 Challenge Setup: Tasks, Datasets and Baseline System

End-to-end Learning for Music Audio Tagging at Scale

Audio Set Classification with Attention Model: A Probabilistic Perspective

mixup: Beyond Empirical Risk Minimization

Transfer learning of weakly labelled audio

Unsupervised Filterbank Learning Using Convolutional Restricted Boltzmann Machine for Environmental Sound Classification

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Transfer Learning for Music Classification and Regression Tasks

Sample-level Deep Convolutional Neural Networks for Music Auto-tagging Using Raw Waveforms

Audio Set: An ontology and human-labeled dataset for audio events

Very deep convolutional neural networks for raw waveforms

CNN architectures for large-scale audio classification

TUT database for acoustic scene classification and sound event detection

Automatic Tagging Using Deep Convolutional Neural Networks

Deep Residual Learning for Image Recognition

ESC: Dataset for Environmental Sound Classification

THE RYERSON AUDIO-VISUAL DATABASE OF EMOTIONAL SPEECH AND SONG

Polyphonic sound event detection using multi label deep neural networks

Detection and Classification of Acoustic Scenes and Events

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Adam: A Method for Stochastic Optimization

A Dataset and Taxonomy for Urban Sound Research

Very Deep Convolutional Networks for Large-Scale Image Recognition

Network In Network

ImageNet classification with deep convolutional neural networks

Recognition of emotional speech for younger and older talkers: Behavioural findings from the toronto emotional speech set

Acoustic event detection in real life recordings

Rectified Linear Units Improve Restricted Boltzmann Machines

ImageNet: A large-scale hierarchical image database

Input-agreement: a new mechanism for collecting data using human computation games

Factors in automatic musical genre classification of audio signals

Classification of general audio data for content-based retrieval

Modeling and classification of natural sounds by product code hidden Markov models

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

ATTENTION-BASED CONVOLUTIONAL NEURAL NETWORK FOR AUDIO EVENT CLASSIFICATION WITH FEATURE TRANSFER LEARNING

Polyphonic Sound Event Detection with Weak Labeling

librosa: Audio and Music Signal Analysis in Python

Transfer Learning by Supervised Pre-training for Audio-based Music Classification

Dropout: a simple way to prevent neural networks from overfitting

AN MFCC-GMM APPROACH FOR EVENT DETECTION AND CLASSIFICATION

NON-SPEECH ENVIRONMENTAL SOUND CLASSIFICATION USING SVMS WITH A NEW SET OF FEATURES

Detecting Alarm Sounds

The blue bars show the number of training clips in logarithmic scale

Class-wise performance of AudioSet tagging systems. Red, blue and black curves are APs of CNN14, MobileNetV1 and the audio tagging system

“DCASE Challenge 2019,”

Field of Study

Journal Information

Name

Page

Volume

Venue Information

Name

Type

URL

Alternate Names