PMAA: A Progressive Multi-scale Attention Autoencoder Model for High-Performance Cloud Removal from Multi-temporal Satellite Imagery

Published in

European Conference on Artificial Intelligence(2023)

External Links:

Generate Graph DownloadPDF

TL;DR

A high-performance cloud removal architecture that concurrently harnesses global and local information to construct robust contextual dependencies using a novel Multi-scale Attention Module (MAM) and a novel Local Interaction Module (LIM) is introduced.

Abstract

Satellite imagery analysis plays a pivotal role in remote sensing; however, information loss due to cloud cover significantly impedes its application. Although existing deep cloud removal models have achieved notable outcomes, they scarcely consider contextual information. This study introduces a high-performance cloud removal architecture, termed Progressive Multi-scale Attention Autoencoder (PMAA), which concurrently harnesses global and local information to construct robust contextual dependencies using a novel Multi-scale Attention Module (MAM) and a novel Local Interaction Module (LIM). PMAA establishes long-range dependencies of multi-scale features using MAM and modulates the reconstruction of fine-grained details utilizing LIM, enabling simultaneous representation of fine- and coarse-grained features at the same level. With the help of diverse and multi-scale features, PMAA consistently outperforms the previous state-of-the-art model CTGAN on two benchmark datasets. Moreover, PMAA boasts considerable efficiency advantages, with only 0.5% and 14.6% of the parameters and computational complexity of CTGAN, respectively. These comprehensive results underscore PMAA's potential as a lightweight cloud removal network suitable for deployment on edge devices to accomplish large-scale cloud removal tasks. Our source code and pre-trained models are available at https://github.com/XavierJiezou/PMAA.

Authors

K. Li

2 papers

Xuechao Zou

2 papers

Junliang Xing

2 papers

References48 items

A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Models

Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition

CTGAN : Cloud Transformer Generative Adversarial Network

An efficient encoder-decoder architecture with top-down attention for speech separation

Reduce Information Loss in Transformers for Pluralistic Image Inpainting

PMAA: A Progressive Multi-scale Attention Autoencoder Model for High-Performance Cloud Removal from Multi-temporal Satellite Imagery

Published in

European Conference on Artificial Intelligence(2023)

External Links:

Generate Graph DownloadPDF

TL;DR

Abstract

Authors

K. Li

2 papers

Xuechao Zou

2 papers

Junliang Xing

2 papers

References48 items

A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Models

Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition

CTGAN : Cloud Transformer Generative Adversarial Network

An efficient encoder-decoder architecture with top-down attention for speech separation

Reduce Information Loss in Transformers for Pluralistic Image Inpainting

Deep Internal Learning for Inpainting of Cloud-Affected Regions in Satellite Imagery

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding

RePaint: Inpainting using Denoising Diffusion Probabilistic Models

Generative Adversarial Networks

Speech Separation Using an Asynchronous Fully Recurrent Convolutional Neural Network

Restormer: Efficient Transformer for High-Resolution Image Restoration

Image Inpainting with External-internal Learning and Monochromic Bottleneck

Multi-Stage Progressive Image Restoration

Thick Clouds Removing From Multitemporal Landsat Images Using Spatiotemporal Neural Networks

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Cloud Removal for Remote Sensing Imagery via Spatial Attention Generative Adversarial Network

Denoising Diffusion Probabilistic Models

Spatial monitoring of grassland management using multi-temporal satellite imagery

FMSSD: Feature-Merged Single-Shot Detection for Multiscale Objects in Large-Scale Remote Sensing Imagery

Survey of single image super-resolution reconstruction

Cloud Removal from Satellite Images using Spatiotemporal Generator Networks

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Cloud Removal of Satellite Images Using Convolutional Neural Network With Reliable Cloudy Image Synthesis Model

RoBERTa: A Robustly Optimized BERT Pretraining Approach

A Framework for Evaluating Land Use and Land Cover Classification Using Convolutional Neural Networks

A Remote Sensing Image Dataset for Cloud Removal

Land-cover classification with high-resolution remote sensing images using transferable deep models

Towards Multi-class Object Detection in Unconstrained Remote Sensing Imagery

A Multi-Temporal Convolutional Autoencoder Neural Network for Cloud Removal in Remote Sensing Images

A Fully Progressive Approach to Single-Image Super-Resolution

Decoupled Weight Decay Regularization

Filmy Cloud Removal on Satellite Imagery with Multispectral Conditional Generative Adversarial Nets

Rethinking Atrous Convolution for Semantic Image Segmentation

Attention is All you Need

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Image-to-Image Translation with Conditional Adversarial Networks

SGDR: Stochastic Gradient Descent with Warm Restarts

Instance Normalization: The Missing Ingredient for Fast Stylization

Deep Residual Learning for Image Recognition

U-Net: Convolutional Networks for Biomedical Image Segmentation

Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies

Rectified Linear Units Improve Restricted Boltzmann Machines

Bayesian Reconstruction of Natural Images from Human Brain Activity

Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders

Image quality assessment: from error visibility to structural similarity

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Field of Study

Computer ScienceEngineering

Journal Information

Name

ArXiv

Volume

abs/2005.00687

Venue Information

Name

European Conference on Artificial Intelligence

Type

conference

URL

http://www.eccai.org/

Alternate Names

ECAI
Eur Conf Artif Intell
International Conference on Electronics, Computers and Artificial Intelligence
Int Conf Electron Comput Artif Intell
International Electronics, Computers and Artificial Intelligence
Int Electron Comput Artif Intell

TL;DR

Abstract

Authors

References48 items

A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Models

Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition

CTGAN : Cloud Transformer Generative Adversarial Network

An efficient encoder-decoder architecture with top-down attention for speech separation

Reduce Information Loss in Transformers for Pluralistic Image Inpainting

TL;DR

Abstract

Authors

References48 items

A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Models

Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition

CTGAN : Cloud Transformer Generative Adversarial Network

An efficient encoder-decoder architecture with top-down attention for speech separation

Reduce Information Loss in Transformers for Pluralistic Image Inpainting

Deep Internal Learning for Inpainting of Cloud-Affected Regions in Satellite Imagery

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding

RePaint: Inpainting using Denoising Diffusion Probabilistic Models

Generative Adversarial Networks

Speech Separation Using an Asynchronous Fully Recurrent Convolutional Neural Network

Restormer: Efficient Transformer for High-Resolution Image Restoration

Image Inpainting with External-internal Learning and Monochromic Bottleneck

Multi-Stage Progressive Image Restoration

Thick Clouds Removing From Multitemporal Landsat Images Using Spatiotemporal Neural Networks

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Cloud Removal for Remote Sensing Imagery via Spatial Attention Generative Adversarial Network

Denoising Diffusion Probabilistic Models

Spatial monitoring of grassland management using multi-temporal satellite imagery

FMSSD: Feature-Merged Single-Shot Detection for Multiscale Objects in Large-Scale Remote Sensing Imagery

Survey of single image super-resolution reconstruction

Cloud Removal from Satellite Images using Spatiotemporal Generator Networks

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Cloud Removal of Satellite Images Using Convolutional Neural Network With Reliable Cloudy Image Synthesis Model

RoBERTa: A Robustly Optimized BERT Pretraining Approach

A Framework for Evaluating Land Use and Land Cover Classification Using Convolutional Neural Networks

A Remote Sensing Image Dataset for Cloud Removal

Land-cover classification with high-resolution remote sensing images using transferable deep models

Towards Multi-class Object Detection in Unconstrained Remote Sensing Imagery

A Multi-Temporal Convolutional Autoencoder Neural Network for Cloud Removal in Remote Sensing Images

A Fully Progressive Approach to Single-Image Super-Resolution

Decoupled Weight Decay Regularization

Filmy Cloud Removal on Satellite Imagery with Multispectral Conditional Generative Adversarial Nets

Rethinking Atrous Convolution for Semantic Image Segmentation

Attention is All you Need

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Image-to-Image Translation with Conditional Adversarial Networks

SGDR: Stochastic Gradient Descent with Warm Restarts

Instance Normalization: The Missing Ingredient for Fast Stylization

Deep Residual Learning for Image Recognition

U-Net: Convolutional Networks for Biomedical Image Segmentation

Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies

Rectified Linear Units Improve Restricted Boltzmann Machines

Bayesian Reconstruction of Natural Images from Human Brain Activity

Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders

Image quality assessment: from error visibility to structural similarity

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Deep Learning

Field of Study

Journal Information

Name

Volume

Venue Information

Name

Type

URL

Alternate Names