Consensual Collaborative Training And Knowledge Distillation Based Facial Expression Recognition Under Noisy Annotations (2021-07-01T00:00:00.000000Z)

TL;DR

This work proposes an effective training strategy in the presence of noisy labels, called as Consensual Collaborative Training (CCT) framework, which co-trains three networks jointly using a convex combination of supervision loss and consistency loss, without making any assumption about the noise distribution.

Abstract

Presence of noise in the labels of large scale facial expression datasets has been a key challenge towards Facial Expression Recognition (FER) in the wild. During early learning stage, deep networks fit on clean data. Then, eventually, they start overfitting on noisy labels due to their memorization ability, which limits FER performance. This work proposes an effective training strategy in the presence of noisy labels, called as Consensual Collaborative Training (CCT) framework. CCT co-trains three networks jointly using a convex combination of supervision loss and consistency loss, without making any assumption about the noise distribution. A dynamic transition mechanism is used to move from supervision loss in early learning to consistency loss for consensus of predictions among networks in the later stage. Inference is done using a single network based on a simple knowledge distillation scheme. Effectiveness of the proposed framework is demonstrated on synthetic as well as real noisy FER datasets. In addition, a large test subset of around 5K images is annotated from the FEC dataset using crowd wisdom of 16 different annotators and reliable labels are inferred. CCT is also validated on it. State-of-the-art performance is reported on the benchmark FER datasets RAFDB (90.84%) FERPlus (89.99%) and AffectNet (66%). Our codes are available at this https URL.

Authors

Darshan Gera

1 papers

B. S

1 papers

TL;DR

Abstract

Authors

References91 items

A Multi-task Mean Teacher for Semi-supervised Facial Affective Behavior Analysis

A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognition

Emotion Recognition with Incomplete Labels Using Modified Multi-task Learning Technique

Multitask Multi-database Emotion Recognition

Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition

Feature Pyramid Network for Multi-task Affective Analysis

An audiovisual and contextual approach for categorical and continuous emotion recognition in-the-wild

Multi-modal Affect Analysis using standardized data within subjects in the Wild

Analysing Affective Behavior in the second ABAW2 Competition

Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study

Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework

THIN: THrowable Information Networks and Application for Facial Expression Recognition in the Wild

Kernelized Dense Layers For Facial Expression Recognition

Noisy Concurrent Training for Efficient Learning under Label Noise

Point Adversarial Self Mining: A Simple Method for Facial Expression Recognition in the Wild

Occlusion-Adaptive Deep Network for Robust Facial Expression Recognition

Deep convolution network based emotion analysis towards mental health care

Semantic Neighborhood-Aware Deep Facial Expression Recognition

Combating Noisy Labels by Agreement: A Joint Training Method with Co-Regularization

Suppressing Uncertainties for Large-Scale Facial Expression Recognition

Analysing Affective Behavior in the First ABAW 2020 Competition

Efficient Facial Feature Learning with Wide Ensemble-based Convolutional Neural Networks

Face Behavior à la carte: Expressions, Affect and Action Units in a Single Network

Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace

NLNL: Negative Learning for Noisy Labels

Symmetric Cross Entropy for Robust Learning With Noisy Labels

MetaCleaner: Learning to Hallucinate Clean Representations for Noisy-Labeled Visual Recognition

Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition

Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism

IMAE for Noise-Robust Learning: Mean Absolute Error Does Not Treat Examples Equally and Gradient Magnitude's Variance Matters

Probabilistic End-To-End Noise Correction for Learning With Noisy Labels

Facial Motion Prior Networks for Facial Expression Recognition

HERO: Human Emotions Recognition for Realizing Intelligent Internet of Things

How does Disagreement Help Generalization against Label Corruption?

Reliable Crowdsourcing and Deep Locality-Preserving Learning for Unconstrained Facial Expression Recognition

A Compact Embedding for Facial Expression Similarity

Generating faces for affect analysis

A Multi-Task Learning & Generation Framework: Valence-Arousal, Action Units & Primary Expressions

Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition

Pumpout: A Meta Approach for Robustly Training Deep Neural Networks with Noisy Labels

Facial Expression Recognition with Inconsistently Annotated Datasets

Covariance Pooling for Facial Expression Recognition

Deep Affect Prediction in-the-Wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond

Local Learning With Deep and Handcrafted Features for Facial Expression Recognition

Deep Facial Expression Recognition: A Survey

Co-teaching: Robust training of deep neural networks with extremely noisy labels

Learning to Reweight Examples for Robust Deep Learning

Iterative Cross Learning on Noisy Labels

MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels

Learning supervised scoring ensemble for emotion recognition in the wild

Coupled Ensembles of Neural Networks

AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild

Recognition of Affect in the Wild Using Deep Neural Networks

Aff-Wild: Valence and Arousal ‘In-the-Wild’ Challenge

Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild

A Closer Look at Memorization in Deep Networks

Decoupling "when to update" from "how to update"

Understanding deep learning requires rethinking generalization

Training deep neural-networks using a noise adaptation layer

RBoost: Label Noise-Robust Boosting Algorithm Based on a Nonconvex Loss Function and the Numerically Stable Base Learners

Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach

Training deep networks for facial expression recognition with crowd-sourced label distribution

MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition

Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks

Deep Residual Learning for Image Recognition

Distilling the Knowledge in a Neural Network

A Confidence-Aware Approach for Truth Discovery on Long-Tail Data

Crowdsourcing for Multiple-Choice Question Answering

Classification in the Presence of Label Noise: A Survey

Human-Computer Interaction Using Emotion Recognition from Facial Expression

Facial expression recognition from near-infrared videos

Facial Expression Measurement for Detecting Driver Drowsiness

The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression

Facial expression recognition based on Local Binary Patterns: A comprehensive study

Performance Comparisons of Facial Expression Recognition in Jaffe Database