Infrared Image Super-Resolution via Transfer Learning and PSRGAN

Abstract

Recent advances in single image super-resolution (SISR) demonstrate the power of deep learning for achieving better performance. Because it is costly to recollect the training data and retrain the model for infrared (IR) image super-resolution, the availability of only a few samples for restoring IR images presents an important challenge in the field of SISR. To solve this problem, we first propose the progressive super-resolution generative adversarial network (PSRGAN) that includes the main path and branch path. The depthwise residual block (DWRB) is used to represent the features of the IR image in the main path. Then, the novel shallow lightweight distillation residual block (SLDRB) is used to extract the features of the readily available visible image in the other path. Furthermore, inspired by transfer learning, we propose the multistage transfer learning strategy for bridging the gap between different high-dimensional feature spaces that can improve the PSGAN performance. Finally, quantitative and qualitative evaluations of two public datasets show that PSRGAN can achieve better results compared to the SR methods.