MAXIM: Multi-Axis MLP for Image Processing (2022-01-09T00:00:00.000000Z)

TL;DR

The proposed MAXIM model achieves state-of-the-art performance on more than ten benchmarks across a range of image processing tasks, including denoising, deblurring, de raining, dehazing, and enhancement while requiring fewer or comparable numbers of parameters and FLOPs than competitive models.

Abstract

Recent progress on Transformers and multilayer perceptron (MLP) models provide new network architectural designs for computer vision tasks. Although these models proved to be effective in many vision tasks such as image recognition, there remain challenges in adapting them for lowlevel vision. The inflexibility to support high-resolution images and limitations of local attention are perhaps the main bottlenecks. In this work, we present a multi-axis MLP based architecture called MAXIM, that can serve as an efficient and flexible general-purpose vision backbone for image processing tasks. MAXIM uses a UNet-shaped hierarchical structure and supports long-range interactions enabled by spatially-gated MLPs. Specifically, MAXIM contains two MLP-based building blocks: a multi-axis gated MLP that allows for efficient and scalable spatial mixing of local and global visual cues, and a cross-gating block, an alternative to cross-attention, which accounts for cross-feature conditioning. Both these modules are exclusively based on MLPs, but also benefit from being both global and ‘fully-convolutional’, two properties that are desirable for image processing. Our extensive experimental results show that the proposed MAXIM model achieves state-of-the-art performance on more than ten benchmarks across a range of image processing tasks, including denoising, deblurring, de raining, dehazing, and enhancement while requiring fewer or comparable numbers of parameters and FLOPs than competitive models. The source code and trained models will be available at https://github.com/google-research/maxim.

Authors

Hossein Talebi

5 papers

P. Milanfar

8 papers

Zhengzhong Tu

6 papers

TL;DR

Abstract

Authors

References123 items

IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information

Learning Enriched Features for Fast Image Restoration and Enhancement

V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer

Toward Real-World Super-Resolution via Adaptive Downsampling Models

SwinIR: Image Restoration Using Swin Transformer

Rethinking Coarse-to-Fine Approach in Single Image Deblurring

Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

Global Filter Networks for Image Classification

Improved Transformer for High-Resolution GANs

Video Super-Resolution Transformer

Uformer: A General U-Shaped Transformer for Image Restoration

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers

Pay Attention to MLPs

HINet: Half Instance Normalization Network for Image Restoration

FNet: Mixing Tokens with Fourier Transforms

ResMLP: Feedforward Networks for Image Classification With Data-Efficient Training

MLP-Mixer: An all-MLP Architecture for Vision

COMISR: Compression-Informed Video Super-Resolution

NTIRE 2021 Challenge on Image Deblurring

Contrastive Learning for Compact Single Image Dehazing

ViViT: A Video Vision Transformer

CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification

Scaling Local Self-Attention for Parameter Efficient Visual Backbones

U-Net Transformer: Self and Cross Attention for Medical Image Segmentation

Multi-Stage Progressive Image Restoration

TransTrack: Multiple-Object Tracking with Transformer

Focal Frequency Loss for Image Reconstruction and Synthesis

Projected Distribution Loss for Image Enhancement

Pre-Trained Image Processing Transformer

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Towards Unsupervised Deep Image Enhancement With Generative Adversarial Network

GIA-Net: Global Information Aware Network for Low-light Imaging

Correction to: Single Image Super-Resolution via a Holistic Attention Network

Real-World Blur Dataset for Learning and Benchmarking Deblurring Algorithms

Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation

End-to-End Object Detection with Transformers

Multi-Scale Boosted Dehazing Network With Dense Feature Fusion

Spatially-Attentive Patch-Hierarchical Network for Adaptive Motion Deblurring

Deblurring by Realistic Blurring

Multi-Scale Progressive Fusion Network for Single Image Deraining

CycleISP: Real Image Restoration via Improved Data Synthesis

Transfer Learning From Synthetic to Real-Noise Denoising With Adaptive Instance Normalization

Spatial-Adaptive Network for Single Image Denoising

Multi-Temporal Recurrent Neural Networks For Progressive Non-Uniform Single Image Deblurring With Incremental Temporal Training

FFA-Net: Feature Fusion Attention Network for Single Image Dehazing

ProxIQA: A Proxy Approach to Perceptual Optimization of Learned Image Compression

Human-Aware Motion Deblurring

Deep Learning for Seeing Through Window With Raindrops

Variational Denoising Network: Toward Blind Noise Modeling and Removal

DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better

GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing

EnlightenGAN: Deep Light Enhancement Without Paired Supervision

Dynamic Scene Deblurring With Parameter Selective Sharing and Nested Skip Connections

Uncertainty Guided Multi-Scale Residual Learning-Using a Cycle Spinning CNN for Single Image De-Raining

Kindling the Darkness: A Practical Low-light Image Enhancer

Real Image Denoising With Feature Attention

Single Image Dehazing with a Generic Model-Agnostic Convolutional Neural Network

Deep Stacked Hierarchical Multi-Patch Network for Image Deblurring

Spatial Attentive Single-Image Deraining With a High Quality Real Rain Dataset

Dual Residual Networks Leveraging the Potential of Paired Operations for Image Restoration

MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation

Deep High-Resolution Representation Learning for Human Pose Estimation

Progressive Image Deraining Networks: A Better and Simpler Baseline

Rethinking on Multi-Stage Networks for Human Pose Estimation

Gated Context Aggregation Network for Image Dehazing and Deraining

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

Deep Retinex Decomposition for Low-Light Enhancement

Semi-Supervised Transfer Learning for Image Rain Removal

CBAM: Convolutional Block Attention Module

Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining

Toward Convolutional Blind Denoising of Real Photographs

Non-Local Recurrent Network for Image Restoration

A High-Quality Denoising Dataset for Smartphone Cameras

Dynamic Scene Deblurring Using Spatially Variant Recurrent Neural Networks

Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs

GLADNet: Low-Light Enhancement Network with Global Awareness