Deep SR-ITM: Joint Learning of Super-Resolution and Inverse Tone-Mapping for 4K UHD HDR Applications (2019-04-25T00:00:00.000000Z)

TL;DR

This paper proposes a joint super-resolution (SR) and inverse tone-mapping (ITM) framework, called Deep SR-ITM, which learns the direct mapping from LR SDR video to their HR HDR version, and shows good subjective quality with increased contrast and details, outperforming the previous joint SR and ITM method.

Abstract

Recent modern displays are now able to render high dynamic range (HDR), high resolution (HR) videos of up to 8K UHD (Ultra High Definition). Consequently, UHD HDR broadcasting and streaming have emerged as high quality premium services. However, due to the lack of original UHD HDR video content, appropriate conversion technologies are urgently needed to transform the legacy low resolution (LR) standard dynamic range (SDR) videos into UHD HDR versions. In this paper, we propose a joint super-resolution (SR) and inverse tone-mapping (ITM) framework, called Deep SR-ITM, which learns the direct mapping from LR SDR video to their HR HDR version. Joint SR and ITM is an intricate task, where high frequency details must be restored for SR, jointly with the local contrast, for ITM. Our network is able to restore fine details by decomposing the input image and focusing on the separate base (low frequency) and detail (high frequency) layers. Moreover, the proposed modulation blocks apply location-variant operations to enhance local contrast. The Deep SR-ITM shows good subjective quality with increased contrast and details, outperforming the previous joint SR-ITM method.

Authors

Jihyong Oh

5 papers

Munchurl Kim

5 papers

S. Kim

4 papers

TL;DR

Abstract

Authors

References47 items

A Multi-purpose Convolutional Neural Network for Simultaneous Super-Resolution and High Dynamic Range Image Reconstruction

Image Super-Resolution Using Very Deep Residual Channel Attention Networks

Recovering Realistic Texture in Image Super-Resolution by Deep Spatial Feature Transform

Deep Back-Projection Networks for Super-Resolution

Residual Dense Network for Image Super-Resolution

HDR image reconstruction from a single exposure using deep CNNs

Advanced High Dynamic Range Imaging

Enhanced Deep Residual Networks for Single Image Super-Resolution

Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution

Learning High Dynamic Range from Outdoor Panoramas

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

Dynamic range expansion based on image statistics

Accurate Image Super-Resolution Using Very Deep Convolutional Networks

U-Net: Convolutional Networks for Biomedical Image Segmentation

Image Super-Resolution Using Deep Convolutional Networks

Adam: A Method for Stochastic Optimization

MatConvNet: Convolutional Neural Networks for MATLAB

High-Quality Reverse Tone Mapping for a Wide Range of Exposures

Physiological inverse tone mapping based on retina response

HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions

Deep Sparse Rectifier Neural Networks

Guided Image Filtering

Understanding the difficulty of training deep feedforward neural networks

Evaluation of reverse tone mapping through varying exposure conditions

High-quality brightness enhancement functions for real-time reverse tone mapping

Ldr2Hdr: on-the-fly reverse tone mapping of legacy video and photographs

A framework for inverse tone mapping

Inverse tone mapping

Interactive local adjustment of tonal values

The Reproduction of Specular Highlights on High Dynamic Range Displays

Image quality assessment: from error visibility to structural similarity

Multiscale structural similarity for image quality assessment

Adaptive Logarithmic Mapping For Displaying High Contrast Scenes

Gradient domain high dynamic range compression

Photographic tone reproduction for digital images

Two methods for display of high contrast images

Image Parameter Values for High Dynamic Range Television for Use in Production and International Programme Exchange. ITU-R Rec

High Dynamic Range Electro-Optical Transfer Function of Mastering Reference Displays

Image Super-resolution

Reference Electro-Optical Transfer Function for Flat Panel Displays used in HDTV Studio Production

Simoncelli and Alan C . Bovik

Values

(i) For single-pass structures, decomposing the input and stacking all three layers

(ii) For two-pass structures, stacking (column (d)) brings 0.35 dB PSNR gain, with decomposed layers

Designing separate passes for the base and detail layers (column (d)) is important with 0.21 dB PSNR gain over column (b)

components of the Deep SR-ITM. Table 3 summarizes the results of this ablation study. The guided filter decomposition and modulation yield 0.08 dB and 0.22 dB

Field of Study

Journal Information

Name

Page

Venue Information

Name

Type

URL

Alternate Names