Imaging through Atmospheric Turbulence

Overview

Imaging through atmospheric turbulence is a fundamental problem for long-range imaging systems. Purdue i2Lab has specialities in several aspects of the subject:

Wave optics theory
Turbulence simulators
Forward imaging models that can be used for deep neural networks
Image reconstruction algorithms
Object recognition

Book

Stanley H. Chan and Nicholas Chimitt, Computational Imaging through Atmospheric Turbulence, Now Publisher 2023.

Tutorial

ICIP 2022 and CVPR 2022
Instructors: Stanley Chan and Nick Chimitt

Recordings

Purdue Atmospheric Turbulence Simulator

Phase-over-aperture model (Version 1)

Key Concept: Collapse screens and sample in the Zernike space. 20x speed up compared to split-step.

Publication:

Nicholas Chimitt and Stanley H. Chan, ‘‘Simulating Anisoplanatic Turbulence by Sampling Correlated Zernike Coefficients’’, Optical Engineering, 59(8), 083101, July 2020. (A shorter version is presented in IEEE ICCP 2020.)

Code:

MATLAB download: (URL)
Python download: (URL)
Licence: Copyright is granted for educational and research purposes. Please contact Prof Chan for licensing.

Project Page: project_turbulence_TurbSim_v1.html

Phase-to-space transform (Version 2)

Key Concept: Transform Zernike representation to PSF representation. 1000x speed up compared to split-step.

Publication:

Zhiyuan Mao, Nicholas Chimitt and Stanley H. Chan, ‘‘Accelerating Atmospheric Turbulence Simulation via Learned Phase-to-Space Transform’’, accepted to IEEE International Conference on Computer Vision, 2021.

Code:

Python download: (URL)
Licence: Copyright is granted for educational and research purposes. Please contact Prof Chan for licensing.

Project Page: project_turbulence_TurbSim_v2.html

Dense field phase-to-space transform (Version 3)

Key concept: Quantize and decouple Zernike mode and spatial mode to preserve wide sense stationarity. Enables full HD without interpolation.

Publication:

Nicholas Chimitt, Xingguang Zhang, Zhiyuan Mao, and Stanley H. Chan, ‘‘Real- Time Dense Field Phase-to-Space Simulation of Imaging through Atmospheric Turbulence’’, IEEE Trans. Computational Imaging, vol. 8, pp. 1159-1169, Dec. 2022.

Arbitrary Cn2 Profile (Version 4)

Key concept: Integrate Cn2 along the path instead of evaluating individual turbulence segments

Publication:

Nicholas Chimitt and Stanley H. Chan, ‘‘Anisoplanatic Optical Turbulence Simulation for Near-Continuous Profiles without Wave Propagation’’, SPIE Optical Engineering (SPIE OpEng), vol. 62, no. 7, pp. 078103, Jul. 2023.

Scattering and Gathering for Spatially Varying Convolutions (Version 5)

Key finding: Scattering is for optics simulation, whereas gathering is for signal filtering

Publication:

Nicholas Chimitt, Xingguang Zhang, Yiheng Chi, Stanley H Chan, ‘‘Scattering and Gathering for Spatially Varying Blurs’’, submitted.

Turbulence Reconstruction

Classical optimization-based approach

Key concept: Lucky imaging + blind deconvolution

Publication:

Zhiyuan Mao, Nicholas Chimitt, and Stanley H. Chan, ‘‘Image Reconstruction of Static and Dynamic Scenes through Anisoplanatic Turbulence’’, IEEE Transactions on Computational Imaging, vol. 6, pp. 1415-1428, Oct. 2020.

MATLAB Code:

MATLAB download: (URL)
Licence: Copyright is granted for educational and research purposes. Please contact Prof Chan for licensing.

[Project Page]

TurbNet: Single-frame Turbulence Reconstruction

Key concept: Re-blur the reconstructed image using a turbulence simulator

Publication:

Zhiyuan Mao, Ajay Jaiswal, Zhangyang Wang, and Stanley H. Chan, ‘‘Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model’’, European Conference on Computer Vision ECCV, 2022.

Code:

Download: [Project Page]

PiRN: Physics-integrated Restoration Network

Key concept: Simulator in the loop

Publication:

Ajay Jaiswal, Xingguang Zhang, Stanley H. Chan, Zhangyang Wang, ‘‘Physics-Driven Turbulence Image Restoration with Stochastic Refinement’’, IEEE International Conference on Computer Vision (ICCV), accepted, 2023.

Code:

Download: [Project Page]

Turbulence Mitigation Transformer

Key concept: Two-stage mitigation, transformer, temporal attention

Publication:

Xingguang Zhang, Zhiyuan Mao, Nicholas Chimitt, and Stanley H. Chan, ‘‘Imaging through the Atmosphere using Turbulence Mitigation Transformer’’, IEEE Transactions on Computational Imaging, Accepted. [Project Page]

Code:

Download: [Project Page]

Theoretical Analysis

Tilt-then-Blur or Blur-then-Tilt

Key conclusion: Tilt + blur is the correct model

Publication:

Stanley H. Chan, ‘‘Tilt-then-Blur or Blur-then-Tilt? Clarifying the Atmospheric Turbulence Model’’, IEEE Signal Processing Letters.