Training a universal instance segmentation network for live cell images of various cell types and imaging modalities (2022-07-28T00:00:00.000000Z)

TL;DR

This work modified the traditional binary training targets to include three classes for direct instance segmentation, and found that segmentation performance can benefit from group normalization layer and Atrous Spatial Pyramid Pooling module, thanks to their more reliable statistics estimation and improved semantic understanding.

Abstract

We share our recent findings in an attempt to train a universal segmentation network for various cell types and imaging modalities. Our method was built on the generalized U-Net architecture, which allows the evaluation of each component individually. We modified the traditional binary training targets to include three classes for direct instance segmentation. Detailed experiments were performed regarding training schemes, training settings, network backbones, and individual modules on the segmentation performance. Our proposed training scheme draws minibatches in turn from each dataset, and the gradients are accumulated before an optimization step. We found that the key to training a universal network is all-time supervision on all datasets, and it is necessary to sample each dataset in an unbiased way. Our experiments also suggest that there might exist common features to define cell boundaries across cell types and imaging modalities, which could allow application of trained models to totally unseen datasets. A few training tricks can further boost the segmentation performance, including uneven class weights in the cross-entropy loss function, well-designed learning rate scheduler, larger image crops for contextual information, and additional loss terms for unbalanced classes. We also found that segmentation performance can benefit from group normalization layer and Atrous Spatial Pyramid Pooling module, thanks to their more reliable statistics estimation and improved semantic understanding, respectively. We participated in the 6th Cell Tracking Challenge (CTC) held at IEEE International Symposium on Biomedical Imaging (ISBI) 2021 using one of the developed variants. Our method was evaluated as the best runner up during the initial submission for the primary track, and also secured the 3rd place in an additional round of competition in preparation for the summary publication.

References78 items

Boosting High-Level Vision with Joint Compression Artifacts Reduction and Super-Resolution

Fibronectin-Expressing Mesenchymal Tumor Cells Promote Breast Cancer Metastasis

Comparison of Backbones for Semantic Segmentation Network

Residual Attention U-Net for Automated Multi-Class Segmentation of COVID-19 Chest CT Images

CycleISP: Real Image Restoration via Improved Data Synthesis

SD-UNet: Stripping down U-Net for Segmentation of Biomedical Images on Platforms with Low Computational Budgets

PyTorch: An Imperative Style, High-Performance Deep Learning Library

J Regularization Improves Imbalanced Multiclass Segmentation

Sea-Land Segmentation With Res-UNet And Fully Connected CRF

Microscale, scanning defocusing volumetric particle-tracking velocimetry

U-Net: deep learning for cell counting, detection, and morphometry

Group Normalization

Residual Dense Network for Image Super-Resolution

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

DensePose: Dense Human Pose Estimation in the Wild

Panoptic Segmentation

Decoupled Weight Decay Regularization

An Objective Comparison of Cell Tracking Algorithms

Deep Residual Networks and Weight Initialization

Neuronal cell-type classification: challenges, opportunities and the path forward

Enhanced Deep Residual Networks for Single Image Super-Resolution

Rethinking Atrous Convolution for Semantic Image Segmentation

Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Mask R-CNN

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

Deep Watershed Transform for Instance Segmentation

Aggregated Residual Transformations for Deep Neural Networks

Xception: Deep Learning with Depthwise Separable Convolutions

Densely Connected Convolutional Networks

SGDR: Stochastic Gradient Descent with Warm Restarts

Instance Normalization: The Missing Ingredient for Fast Stylization

Layer Normalization

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

Cell Tracking Accuracy Measurement Based on Comparison of Acyclic Oriented Graphs

Deep Residual Learning for Image Recognition

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

Proposal-Free Network for Instance-Level Object Segmentation

You Only Look Once: Unified, Real-Time Object Detection

U-Net: Convolutional Networks for Biomedical Image Segmentation

Learning Deconvolution Network for Semantic Segmentation

FlowNet: Learning Optical Flow with Convolutional Networks

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Striving for Simplicity: The All Convolutional Net

Hypercolumns for object segmentation and fine-grained localization

Fully convolutional networks for semantic segmentation

Very Deep Convolutional Networks for Large-Scale Image Recognition

ImageNet Large Scale Visual Recognition Challenge

Objective comparison of particle tracking methods

Rectified Linear Units Improve Restricted Boltzmann Machines

Deconvolutional networks

Algorithms

Bed Exit Detection Network (BED Net) for Patients Bed-Exit Monitoring Based on Color Camera Images

Remote estimation of respiration rate by optical flow using convolutional neural networks

Multi-parametric feature-based cell tracking in timelapse microscopy [manuscript in preparation

“Micro-Batch Training with Batch-Channel Normalization and Weight Standardization,”

Hinton

“Empowering Faculty: A Campus Cyberinfrastructure Strategy for Research Communities,”

Learning Multiple Layers of Features from Tiny Images

Morphological segmentation revisited

Computing global shape measures

Conference on Computer Vision and Pattern Recognition

Distributed under Creative Commons Cc-by 4.0 Scikit-image: Image Processing in Python

Convstride: We replaced the 2 × 2 maxpooling layers in the encoder with 3 × 3 convolutional layers with stride of 2 [54], which allows learnable downsampling of the feature maps

Instance segmentation can be achieved by joint-training with an object detection network

Crop: A random crop of 512 × 512 pixels is used for datasets whose shorter image side length is at least 512 pixels, while datasets with smaller images still use random

Although the ﬁnal cross-entropy loss of the Lfcn variant is slightly higher than the four cross-entropy-only variants as shown in

What is a better way to train a universal cell segmentation network that works for various cell types and imaging modalities? 2.3.

Is it possible to achieve comparable or even better segmentation performance if we use other backbones to replace the VGG-like encoder in the original U-Net

corresponds to equally 2.6k minibatches drawn from each dataset, except for the scheme Mix

The cosine annealing learning rate scheduler [39] without the warm restarts is used. The max and min learning rates are set to 2 × 10 − 4 and 1 × 10 − 6

At each iteration, randomly choose one dataloader to draw a minibatch

all 13 validation sets we select the best models to report. 4.1.2.

: Besides the cross-entropy loss, we use an additional J-regularization loss [11] to deal with unbalanced classes. The summation of the two losses with equal weights are jointly optimized

: Minibatches are drawn in turn from each dataloader, in the pre-deﬁned ﬁxed order