CLUSTSEG: Clustering for Universal Segmentation

Published in

International Conference on Machine Learning(2023)

External Links:

Generate Graph DownloadPDF

TL;DR

These innovations closely link CLUSTSEG to EM clustering and make it a transparent and powerful framework that yields superior results across the above segmentation tasks.

Abstract

We present CLUSTSEG, a general, transformer-based framework that tackles different image segmentation tasks (i.e., superpixel, semantic, instance, and panoptic) through a unified neural clustering scheme. Regarding queries as cluster centers, CLUSTSEG is innovative in two aspects:1) cluster centers are initialized in heterogeneous ways so as to pointedly address task-specific demands (e.g., instance- or category-level distinctiveness), yet without modifying the architecture; and 2) pixel-cluster assignment, formalized in a cross-attention fashion, is alternated with cluster center update, yet without learning additional parameters. These innovations closely link CLUSTSEG to EM clustering and make it a transparent and powerful framework that yields superior results across the above segmentation tasks.

Authors

Wenguan Wang

11 papers

J. Liang

1 papers

Tianfei Zhou

2 papers

References92 items

Tripartite Feature Enhanced Pyramid Network for Dense Prediction

Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks

GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models

Learning Equivariant Segmentation with Instance-Unique Querying

Visual Recognition with Deep Nearest Centroids

CLUSTSEG: Clustering for Universal Segmentation

Published in

International Conference on Machine Learning(2023)

External Links:

Generate Graph DownloadPDF

TL;DR

These innovations closely link CLUSTSEG to EM clustering and make it a transparent and powerful framework that yields superior results across the above segmentation tasks.

Abstract

Authors

Wenguan Wang

11 papers

J. Liang

1 papers

Tianfei Zhou

2 papers

References92 items

Tripartite Feature Enhanced Pyramid Network for Dense Prediction

Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks

GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models

Learning Equivariant Segmentation with Instance-Unique Querying

Visual Recognition with Deep Nearest Centroids

Dongfang Liu

1 papers

Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patches

CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation

A Triangulation-Based Visual Localization for Field Robots

Sparse Instance Activation for Real-Time Instance Segmentation

A ConvNet for the 2020s

Masked-attention Mask Transformer for Universal Image Segmentation

Multi-Scale High-Resolution Vision Transformer for Semantic Segmentation

Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers

SOTR: Segmenting Objects with Transformers

A Unified Efficient Pyramid Transformer for Semantic Segmentation

Per-Pixel Classification is Not All You Need for Semantic Segmentation

K-Net: Towards Unified Image Segmentation

SOLQ: Segmenting Objects by Learning Queries

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers

Segmenter: Transformer for Semantic Segmentation

Instances as Queries

ISTR: End-to-End Instance Segmentation with Transformers

Learning the Superpixel in a Non-iterative and Lifelong Manner

SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentation

Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing

Exploring Cross-Image Pixel Contrast for Semantic Segmentation

Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers

DenserNet: Weakly Supervised Visual Localization Using Multi-scale Feature Aggregation

Fully Convolutional Networks for Panoptic Segmentation

MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers

End-to-End Object Detection with Transformers

Pixel Consensus Voting for Panoptic Segmentation

Superpixel Segmentation With Fully Convolutional Networks

SOLOv2: Dynamic and Fast Instance Segmentation

Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation

Unifying Training and Inference for Panoptic Segmentation

BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation

PointRend: Image Segmentation As Rendering

SOLO: Segmenting Objects by Locations

Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation

Cascade R-CNN: High Quality Object Detection and Instance Segmentation

MMDetection: Open MMLab Detection Toolbox and Benchmark

Learning Instance Occlusion for Panoptic Segmentation

Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth

Adaptive Pyramid Context Network for Semantic Segmentation

TensorMask: A Foundation for Dense Object Segmentation

An End-To-End Network for Panoptic Segmentation

Mask Scoring R-CNN

Hybrid Task Cascade for Instance Segmentation

UPSNet: A Unified Panoptic Segmentation Network

Panoptic Feature Pyramid Networks

Attention-Guided Unified Network for Panoptic Segmentation

Dual Attention Network for Scene Segmentation

PSANet: Point-wise Spatial Attention Network for Scene Parsing

Superpixel Sampling Networks

Learning Superpixels with Segmentation-Aware Affinity Loss

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Panoptic Segmentation

Recurrent Pixel Embedding for Instance Grouping

MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features

Non-local Neural Networks

Squeeze-and-Excitation Networks

Segmentation-Aware Convolutional Networks Using Local Attention Masks

Scene Parsing through ADE20K Dataset

Attention is All you Need

Superpixels: An evaluation of the state-of-the-art

Deep Watershed Transform for Instance Segmentation

InstanceCut: From Edges to Instances with MultiCut

Categorical Reparameterization with Gumbel-Softmax

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Deep Residual Learning for Image Recognition

Multi-Scale Context Aggregation by Dilated Convolutions

U-Net: Convolutional Networks for Biomedical Image Segmentation

Conditional Random Fields as Recurrent Neural Networks

Fully convolutional networks for semantic segmentation

Deeply-Supervised Nets

Microsoft COCO: Common Objects in Context

SLIC Superpixels Compared to State-of-the-Art Superpixel Methods

Indoor Segmentation and Support Inference from RGBD Images

A Comparative Study of Efficient Initialization Methods for the K-Means Clustering Algorithm

Contour Detection and Hierarchical Image Segmentation

ImageNet: A large-scale hierarchical image database

k-means Requires Exponentially Many Iterations Even in the Plane

Cluster center initialization algorithm for K-means clustering

Alternatives to the k-means algorithm that find better clusterings

Image segmentation by clustering

k-means Mask Transformer

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Qualitative semantic segmentation results on ADE20K

ADE20K/) is released under a CC BSD-3; All assets mentioned above release annotations obtained from human experts with agreements

Clustering for Universal Segmentation

Field of Study

Computer Science

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

International Conference on Machine Learning

Type

conference

URL

https://icml.cc/

Alternate Names

ICML
Int Conf Mach Learn

TL;DR

Abstract

Authors

References92 items

Tripartite Feature Enhanced Pyramid Network for Dense Prediction

Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks

GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models

Learning Equivariant Segmentation with Instance-Unique Querying

Visual Recognition with Deep Nearest Centroids

TL;DR

Abstract

Authors

References92 items

Tripartite Feature Enhanced Pyramid Network for Dense Prediction

Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks

GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models

Learning Equivariant Segmentation with Instance-Unique Querying

Visual Recognition with Deep Nearest Centroids

Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patches

CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation

A Triangulation-Based Visual Localization for Field Robots

Sparse Instance Activation for Real-Time Instance Segmentation

A ConvNet for the 2020s

Masked-attention Mask Transformer for Universal Image Segmentation

Multi-Scale High-Resolution Vision Transformer for Semantic Segmentation

Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers

SOTR: Segmenting Objects with Transformers

A Unified Efficient Pyramid Transformer for Semantic Segmentation

Per-Pixel Classification is Not All You Need for Semantic Segmentation

K-Net: Towards Unified Image Segmentation

SOLQ: Segmenting Objects by Learning Queries

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers

Segmenter: Transformer for Semantic Segmentation

Instances as Queries

ISTR: End-to-End Instance Segmentation with Transformers

Learning the Superpixel in a Non-iterative and Lifelong Manner

SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentation

Differentiable Multi-Granularity Human Representation Learning for Instance-Aware Human Semantic Parsing

Exploring Cross-Image Pixel Contrast for Semantic Segmentation

Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers

DenserNet: Weakly Supervised Visual Localization Using Multi-scale Feature Aggregation

Fully Convolutional Networks for Panoptic Segmentation

MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers

End-to-End Object Detection with Transformers

Pixel Consensus Voting for Panoptic Segmentation

Superpixel Segmentation With Fully Convolutional Networks

SOLOv2: Dynamic and Fast Instance Segmentation

Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation

Unifying Training and Inference for Panoptic Segmentation

BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation

PointRend: Image Segmentation As Rendering

SOLO: Segmenting Objects by Locations

Panoptic-DeepLab: A Simple, Strong, and Fast Baseline for Bottom-Up Panoptic Segmentation

Cascade R-CNN: High Quality Object Detection and Instance Segmentation

MMDetection: Open MMLab Detection Toolbox and Benchmark

Learning Instance Occlusion for Panoptic Segmentation

Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth

Adaptive Pyramid Context Network for Semantic Segmentation

TensorMask: A Foundation for Dense Object Segmentation

An End-To-End Network for Panoptic Segmentation

Mask Scoring R-CNN

Hybrid Task Cascade for Instance Segmentation

UPSNet: A Unified Panoptic Segmentation Network

Panoptic Feature Pyramid Networks

Attention-Guided Unified Network for Panoptic Segmentation

Dual Attention Network for Scene Segmentation

PSANet: Point-wise Spatial Attention Network for Scene Parsing

Superpixel Sampling Networks

Learning Superpixels with Segmentation-Aware Affinity Loss

Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation

Panoptic Segmentation

Recurrent Pixel Embedding for Instance Grouping

MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features

Non-local Neural Networks

Squeeze-and-Excitation Networks

Segmentation-Aware Convolutional Networks Using Local Attention Masks

Scene Parsing through ADE20K Dataset

Attention is All you Need

Mask R-CNN

Superpixels: An evaluation of the state-of-the-art