L1BSR: Exploiting Detector Overlap for Self-Supervised Single-Image Super-Resolution of Sentinel-2 L1B Imagery (2023-04-14T00:00:00.000000Z)

TL;DR

This work proposes L1BSR, a deep learning-based method for single-image super-resolution and band alignment of Sentinel-2 L1B 10m bands and demonstrates the performance of the proposed approach on synthetic and real L 1B data, where it obtains comparable results to supervised methods.

Abstract

High-resolution satellite imagery is a key element for many Earth monitoring applications. Satellites such as Sentinel-2 feature characteristics that are favorable for super-resolution algorithms such as aliasing and band-misalignment. Unfortunately the lack of reliable high-resolution (HR) ground truth limits the application of deep learning methods to this task. In this work we propose L1BSR, a deep learning-based method for single-image super-resolution and band alignment of Sentinel-2 L1B 10m bands. The method is trained with self-supervision directly on real L1B data by leveraging overlapping areas in L1B images produced by adjacent CMOS detectors, thus not requiring HR ground truth. Our self-supervised loss is designed to enforce the super-resolved output image to have all the bands correctly aligned. This is achieved via a novel cross-spectral registration network (CSR) which computes an optical flow between images of different spectral bands. The CSR network is also trained with self-supervision using an Anchor-Consistency loss, which we also introduce in this work. We demonstrate the performance of the proposed approach on synthetic and real L1B data, where we show that it obtains comparable results to supervised methods.

Authors

N. Nguyen

1 papers

J. Anger

2 papers

Axel Davy

3 papers

TL;DR

Abstract

Authors

References50 items

Handheld Burst Super-Resolution Meets Multi-Exposure Satellite Imagery

On The Role of Alias and Band-Shift for Sentinel-2 Super-Resolution

SEN2VENµS, a Dataset for the Training of Sentinel-2 Super-Resolution Algorithms

Super-Resolution of Sentinel-2 Images Using a Spectral Attention Mechanism

RFNet: Unsupervised Network for Mutually Reinforcing Multi-modal Image Registration and Fusion

Self-Supervised Super-Resolution for Multi-Exposure Push-Frame Satellites

Unsupervised Real-World Super-Resolution: A Domain Adaptation Perspective

A review of multimodal image matching: Methods and applications

Self-supervised multi-image super-resolution for push-frame satellite images

There and Back Again: Self-supervised Multispectral Correspondence Estimation

Unsupervised Deep Video Denoising

Super-Resolution of Sentinel-2 Images Using Convolutional Neural Networks and Real Ground Truth Data

FAST AND ACCURATE MULTI-FRAME SUPER-RESOLUTION OF SATELLITE IMAGES

Super-Resolution of Sentinel-2 Imagery Using Generative Adversarial Networks

CoMIR: Contrastive Multimodal Image Representation for Registration

Joint Learning of Blind Video Denoising and Optical Flow Estimation

Multi-Image Super-Resolution for Remote Sensing using Deep Recurrent Networks

Unsupervised Real-World Super Resolution with Cycle Generative Adversarial Network and Domain Discriminator

Self-supervised training for blind multi-frame video denoising

Boosting Structure Consistency for Multispectral and Multimodal Image Registration

Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation

A GENERATIVE ADVERSARIAL NETWORK APPROACH FOR SUPER-RESOLUTION OF SENTINEL-2 SATELLITE IMAGES

Meta-Transfer Learning for Zero-Shot Super-Resolution

HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery

Applicable

Fast Super-Resolution of 20 m Sentinel-2 Bands Using Convolutional Neural Networks

Unsupervised Learning for Real-World Super-Resolution

Super-resolution of PROBA-V images using convolutional neural networks

Joint demosaicing and denoising by overfitting of bursts of raw images

Toward Real-World Single Image Super-Resolution: A New Benchmark and a New Model

Deep Learning for Image Super-Resolution: A Survey

Model-Blind Video Denoising via Frame-To-Frame Training

Image Super-Resolution Using Very Deep Residual Channel Attention Networks

Unsupervised Image Super-Resolution Using Cycle-in-Cycle Generative Adversarial Networks

Mutual information based multi-modal remote sensing image registration using adaptive feature weight

A Novel Affine and Contrast Invariant Descriptor for Infrared and Visible Image Registration

Super-Resolution of Sentinel-2 Images: Learning a Globally Applicable Deep Neural Network

Noise2Noise: Learning Image Restoration without Clean Data

Zero-Shot Super-Resolution Using Deep Internal Learning

Squeeze-and-Excitation Networks

Copernicus Sentinel-2A Calibration and Products Validation Status

Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity

Image-to-Image Translation with Conditional Adversarial Networks

U-Net: Convolutional Networks for Biomedical Image Segmentation

Adam: A Method for Stochastic Optimization

TV-L1 Optical Flow Estimation

Sentinel-2: ESA's Optical High-Resolution Mission for GMES Operational Services

Understanding the difficulty of training deep feedforward neural networks

Learning similarity measure for multi-modal 3D image registration

Limits on super-resolution and how to break them

Field of Study

Journal Information

Name

Page