Robust Learning from Untrusted Sources (2019-01-29T00:00:00.000000Z)

TL;DR

This work derives a procedure that allows for learning from all available sources, yet automatically suppresses irrelevant or corrupted data, and shows that this method provides significant improvements over alternative approaches from robust statistics and distributed optimization.

Abstract

Modern machine learning methods often require more data for training than a single expert can provide. Therefore, it has become a standard procedure to collect data from external sources, e.g. via crowdsourcing. Unfortunately, the quality of these sources is not always guaranteed. As additional complications, the data might be stored in a distributed way, or might even have to remain private. In this work, we address the question of how to learn robustly in such scenarios. Studying the problem through the lens of statistical learning theory, we derive a procedure that allows for learning from all available sources, yet automatically suppresses irrelevant or corrupted data. We show by extensive experiments that our method provides significant improvements over alternative approaches from robust statistics and distributed optimization.

Authors

Christoph H. Lampert

5 papers

Nikola Konstantinov

1 papers

TL;DR

Abstract

Authors

References52 items

Improving the Generalization of Adversarial Training with Domain Adaptation

Mitigating Sybils in Federated Learning Poisoning

Algorithms and Theory for Multiple-Source Adaptation

KS(conf ): A Light-Weight Test if a ConvNet Operates Outside of Its Specifications

Byzantine Stochastic Gradient Descent

The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory

Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates

Robust estimation via robust gradient estimation

Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise

Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent

Learning Discrete Distributions from Untrusted Batches

Multi-view learning overview: Recent progress and new challenges

Practical Secure Aggregation for Privacy-Preserving Machine Learning

Robust Multi-view Topic Modeling by Incorporating Detecting Anomalies

Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly

Federated Multi-Task Learning

A Simple but Tough-to-Beat Baseline for Sentence Embeddings

Learning Theory for Conditional Risk Minimization

On Fundamental Limits of Robust Learning

Efficient PAC Learning from the Crowd

Learning from untrusted data

Intel® Software Guard Extensions (Intel® SGX) Support for Dynamic Memory Management Inside an Enclave

Robust Estimators in High Dimensions without the Computational Intractability

Multi-task Learning with Labeled and Unlabeled Tasks

Communication-Efficient Learning of Deep Networks from Decentralized Data

Deep Residual Learning for Image Recognition

Privacy-preserving deep learning

Inferring Networks of Substitutable and Complementary Products

Taming the Wild: A Unified Analysis of Hogwild-Style Algorithms

Image-Based Recommendations on Styles and Substitutes

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

GloVe: Global Vectors for Word Representation

Distributed Robust Learning

ImageNet Large Scale Visual Recognition Challenge

Learning to Predict from Crowdsourced Data

Robust domain adaptation

Generalization Bounds for Domain Adaptation

Large Scale Distributed Deep Networks

A Convex Formulation for Learning from Crowds

Poisoning Attacks against Support Vector Machines

New Analysis and Algorithm for Learning with Drifting Distributions

A theory of learning from different domains

Domain Adaptation with Multiple Sources

Learning from Multiple Sources

Introduction to Statistical Learning Theory

Resistant fits for some commonly used logistic models with medical application.

Robust Learning from Multiple Information Sources

Understanding Machine Learning - From Theory to Algorithms

Towards Building a High-Quality Workforce with Mechanical

Trusted Computing, Second International Conference, Trust 2009, Oxford, UK, April 6-8, 2009, Proceedings

Different data from different labs: lessons from studies of gene-environment interaction.

A survey of sampling from contaminated distributions

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names