Accelerating Neural Style-Transfer Using Contrastive Learning for Unsupervised Satellite Image Super-Resolution

Divya Mishra, Ofer Hadar

Research output: Contribution to journalArticlepeer-review

Abstract

Contrastive learning is a self-supervised comparison of two samples to identify characteristics and traits that distinguish one data class from another, improving performance on visual tasks. The performance of existing super-resolution-based approaches degrades with increasing scaling factors, hence practically not useful for high-resolution (HR) imaging applications. We proposed a novel framework that uses contrastive training followed by a decoder to generate an 'Artificial style image,' which is utilized as a style image for neural style transfer (NST) learning for image super-resolution in an unsupervised manner. The idea is to benefit from HR textures and features as a style and transfer on an original low-resolution (LR) content image as base elements. The proposed framework has three benefits: 1) the framework is capable of super-resolving different modalities of data like single-band remote sensing images, multispectral band images, RGB remote sensing images, and real-world natural images; 2) proposed method outperforms existing unsupervised and also supervised learning-based methods for both visual and qualitative performance; and 3) leveraging NST learning for remote sensing image super-resolution is performed without sacrificing speed and resources. The framework is novel since the work on NST learning to super-resolve remote sensing images in an unsupervised manner has yet to be acknowledged.

Original languageEnglish
Article number4705014
JournalIEEE Transactions on Geoscience and Remote Sensing
Volume61
DOIs
StatePublished - 1 Jan 2023

Keywords

  • Contrastive learning
  • deep learning
  • style transfer learning
  • unsupervised image super-resolution

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • General Earth and Planetary Sciences

Fingerprint

Dive into the research topics of 'Accelerating Neural Style-Transfer Using Contrastive Learning for Unsupervised Satellite Image Super-Resolution'. Together they form a unique fingerprint.

Cite this