computer-vision-10

Material Segmentation

3260 papers • 126 benchmarks • 313 datasets

Segment materials in a image

(Image credit: Papersgraph)

Benchmarks

These leaderboards are used to track progress in material-segmentation-10

Trend

Dataset

Best Model

Actions

No benchmarks available.

Libraries

i

Use these libraries to find material-segmentation-10 models and implementations

Datasets

No datasets available.

Subtasks

No subtasks available.

Most implemented papers

A Dense Material Segmentation Dataset for Indoor and Outdoor Scene Parsing

Paul Upchurch, Ransen Niu•Wed Jul 20 2022

It is shown that a model trained on the data outperforms a state-of-the-art model across datasets and viewpoints, and is proposed to address this with a large-scale dataset of 3.2 million dense segments on 44,560 indoor and outdoor images.

31

Content

0

Paper Graph

Multimodal Material Segmentation

Yu-Shan Liang, Ryosuke Wakaki, S. Nobuhara, K. Nishino•Tue May 31 2022

Recognition of materials from their visual appearance is essential for computer vision tasks, especially those that involve interaction with the real world. Material segmentation, i.e., dense per-pixel recognition of materials, remains challenging as, unlike objects, materials do not exhibit clearly discernible visual signatures in their regular RGB appearances. Different materials, however, do lead to different radiometric behaviors, which can often be captured with non-RGB imaging modalities. We realize multimodal material segmentation from RGB, polarization, and near-infrared images. We introduce the MCubeS dataset (from MultiModal Material Segmentation) which contains 500 sets of multimodal images capturing 42 street scenes. Ground truth material segmentation as well as semantic segmentation are annotated for every image and pixel. We also derive a novel deep neural network, MCubeSNet, which learns to focus on the most informative combinations of imaging modalities for each material class with a newly derived region-guided filter selection (RGFS) layer. We use semantic segmentation as a prior to “guide” this filter selection. To the best of our knowledge, our work is the first comprehensive study on truly multimodal material segmentation. We believe our work opens new avenues of practical use of material information in safety critical applications.

77 0

Paper Graph

MatSpectNet: Material Segmentation Network with Domain-Aware and Physically-Constrained Hyperspectral Reconstruction

Jiawen Chen, Yuwen Heng, S. Dasmahapatra, Hansung Kim, Yihong Wu•Thu Jul 20 2023

A novel model is proposed, the MatSpectNet to segment materials with recovered hyperspectral images from RGB images, which leverages the principles of colour perception in modern cameras to constrain the reconstructed hyperspectral images and employs the domain adaptation method to generalise the hyperspectral reconstruction capability from a spectral recovery dataset to material segmentation datasets.

2 0

Paper Graph

Beyond Appearances: Material Segmentation with Embedded Spectral Information from RGB-D imagery

Fabian Perez, H. Rueda-Chacón•Sun Jun 16 2024

In the realm of computer vision, material segmentation of natural scenes represents a challenge, driven by the complex and diverse appearances of materials. Traditional approaches often rely on RGB images, which can be deceptive given the variability in appearances due to different lighting conditions. Other methods, that employ polarization or spectral imagery, offer a more reliable material differentiation but their cost and accessibility restrict their everyday usage. In this work, we propose a deep learning framework that bridges the gap between high-fidelity material segmentation and the practical constraints of data acquisition. Our approach leverages a training strategy that employs a paired RGBD-spectral data to incorporate spectral information directly within the neural network. This encoding process is facilitated by a Spectral Feature Mapper (SFM) layer, a novel module that embeds unique spectral characteristics into the network, thus enabling the network to infer materials from standard RGB-D images. Once trained, the model allows to conduct material segmentation on widely available devices without the need for direct spectral data input. In addition, we generate the 3D point cloud from the RGB-D image pair, to provide a richer spatial context for scene understanding. Through simulations using available datasets, and real experiments conducted with an iPad Pro, our method demonstrates superior performance in material segmentation compared to other methods. Code is available at: https://github.com/Factral/Spectral-material-segmentation

2 0

Paper Graph

DBAT: Dynamic Backward Attention Transformer for Material Segmentation with Cross-Resolution Patches

Yuwen Heng, S. Dasmahapatra, Hansung Kim•Fri May 05 2023

By analysing the cross-resolution features and the attention weights, this paper interprets how the DBAT learns from image patches and shows that the proposed model can extract material-related features better than other methods.

1 0

Paper Graph

Adding a benchmark result helps the community track progress.