Atmospheric Turbulence Degraded Video Restoration with Recurrent GAN (ATVR-GAN)

Bar Ettedgui, Yitzhak Yitzhaky

Research output: Contribution to journalArticlepeer-review

Abstract

Atmospheric turbulence (AT) can change the path and direction of light during video capturing of a target in space due to the random motion of the turbulent medium, a phenomenon that is most noticeable when shooting videos at long ranges, resulting in severe video dynamic distortion and blur. To mitigate geometric distortion and reduce spatially and temporally varying blur, we propose a novel Atmospheric Turbulence Video Restoration Generative Adversarial Network (ATVR-GAN) with a specialized Recurrent Neural Network (RNN) generator, which is trained to predict the scene's turbulent optical flow (OF) field and utilizes a recurrent structure to catch both spatial and temporal dependencies. The new architecture is trained using a newly combined loss function that counts for the spatiotemporal distortions, specifically tailored to the AT problem. Our network was tested on synthetic and real imaging data and compared against leading algorithms in the field of AT mitigation and image restoration. The proposed method outperformed these methods for both synthetic and real data examined.

Original languageAmerican English
JournalSensors
Volume23
Issue number21
DOIs
StatePublished - 30 Oct 2023

Keywords

  • CNN
  • GAN
  • RNN
  • atmospheric turbulence
  • optical flow
  • video restoration

All Science Journal Classification (ASJC) codes

  • Analytical Chemistry
  • Information Systems
  • Instrumentation
  • Atomic and Molecular Physics, and Optics
  • Electrical and Electronic Engineering
  • Biochemistry

Fingerprint

Dive into the research topics of 'Atmospheric Turbulence Degraded Video Restoration with Recurrent GAN (ATVR-GAN)'. Together they form a unique fingerprint.

Cite this