Imperfect Segmentation Labels: How Much Do They Matter? (2018-06-12T00:00:00.000000Z)

TL;DR

This large-scale study of model performance in the presence of varying types and degrees of error in training data shows that for each architecture, performance steadily declines with boundary-localized errors, however, U-Net was significantly more robust to jagged boundary errors than the other architectures.

Abstract

Labeled datasets for semantic segmentation are imperfect, especially in medical imaging where borders are often subtle or ill-defined. Little work has been done to analyze the effect that label errors have on the performance of segmentation methodologies. Here we present a large-scale study of model performance in the presence of varying types and degrees of error in training data. We trained U-Net, SegNet, and FCN32 several times for liver segmentation with 10 different modes of ground-truth perturbation. Our results show that for each architecture, performance steadily declines with boundary-localized errors, however, U-Net was significantly more robust to jagged boundary errors than the other architectures. We also found that each architecture was very robust to non-boundary-localized errors, suggesting that boundary-localized errors are fundamentally different and more challenging problem than random label errors in a classification setting.

Authors

N. Heller

4 papers

Joshua Dean

3 papers

N. Papanikolopoulos

3 papers

Imperfect Segmentation Labels: How Much Do They Matter?

TL;DR

Abstract

Authors

References26 items

The potential for machine learning algorithms to improve and reduce the cost of 3-dimensional printing for surgical planning

An application of cascaded 3D fully convolutional networks for medical image segmentation

A Web-Based Platform for Distributed Annotation of Computerized Tomography Scans

Liver Tumor Segmentation from MR Images Using 3D Fast Marching Algorithm and Single Hidden Layer Feedforward Neural Network

V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

Deep learning for visual understanding: A review

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation

Learning from massive noisy labeled data for image classification

U-Net: Convolutional Networks for Biomedical Image Segmentation

Adam: A Method for Stochastic Optimization

Training Deep Neural Networks on Noisy Labels with Bootstrapping

Fully convolutional networks for semantic segmentation

Classification in the Presence of Label Noise: A Survey

Automatic

Deep Feature Learning for Knee Cartilage Segmentation Using a Triplanar Convolutional Neural Network

Introduction to Semi-Supervised Learning

Ensemble Methods for Noise Elimination in Classification Problems

Identifying Mislabeled Training Data

Learning From Noisy Examples

Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis

Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis

Keras

Relabeling

This Paper Is Included in the Proceedings of the 12th Usenix Symposium on Operating Systems Design and Implementation (osdi '16). Tensorflow: a System for Large-scale Machine Learning Tensorflow: a System for Large-scale Machine Learning

Imperfect Segmentation Labels: How Much Do They Matter? 9

Field of Study