Space-time Fourier ptychography for in vivo quantitative phase imaging

Ming Sun, Kunyi Wang, Yogeshwar Nath Mishra, Simeng Qiu, Wolfgang Heidrich*
King Abdullah University of Science and Technology (KAUST)
Optica
Paper (Optica) Paper (COSI 2024) Code (to be released)

Comparative Results of Live Tardigrade Imaging: Showing raw data, zoomed ROI, amplitude results from Compressive Sensing based Fourier Ptychography (CS-FP) versus Space-Time Fourier Ptychography (ST-FP), and phase reconstructions.

Abstract

Quantitative phase imaging of living biological specimens is challenging due to their continuous movement and complex behavior. Here, we introduce Space-time Fourier ptychography (ST-FP), which combines a fast Fourier ptychography (FP) model based on compressive sensing with novel space-time motion priors for joint reconstruction of quantitative phase, intensity, and motion fields across consecutive frames. Using the same input data as compressed sensing FP, ST-FP increases the space-bandwidth-time product of the reconstructed complex image sequences while leveraging redundant temporal information to achieve robust reconstruction performance. The efficacy of this approach is demonstrated across various applications, particularly in observing living microorganisms undergoing rapid morphological changes and reconstructing amplitude and phase targets in motion. The improved phase retrieval capability of ST-FP enables digital refocusing, facilitating comprehensive three-dimensional analysis of microorganisms. This advancement paves the way for enhanced visualization of cellular processes, developmental biology studies, and investigations into life mechanics at the microscopic level.

Methodology

MY ALT TEXT

Principle of Space-time Fourier Ptychography (ST-FP). (a) Schematic of ST-FP setup, including multiplexed LED illumination and the pathway through lenses to the sensor, captures the dynamics of living organisms. (b) To address rapid deformations between successive frames, a novel reconstruction approach utilizes warping to approximate intermediate frame states. For each time stamp, raw data bt is captured under multiplexed illumination. Utilizing motion fields -vt-1 and vt, backward and forward warping are applied to estimate ot-1 and ot+1, respectively. This approach effectively aggregates phase and amplitude information across time, resulting in increased reconstruction accuracy and resolution. This method aligns each captured raw frame with its temporal stamp, allowing for the reconstruction of objects even with significant motion, as opposed to the traditional scheme that requires negligible object movement.

Real Experiments with Controlled Motion

-->

Media Results (Moving Targets & Live Cells)

Poster (ICCP 2024, Spotlight)

BibTeX

@article{Sun:Optica24,
author = {Ming Sun and Kunyi Wang and Yogeshwar Nath Mishra and Simeng Qiu and Wolfgang Heidrich},
journal = {Optica},
number = {9},
pages = {1250--1260},
title = {Space-time Fourier ptychography for in vivo quantitative phase imaging},
volume = {11},
month = {Sep},
year = {2024},
url = {https://opg.optica.org/optica/abstract.cfm?URI=optica-11-9-1250},
doi = {10.1364/OPTICA.531646},
}

BibTeX

@inproceedings{Sun:COSI24,
author = {Ming Sun and Kunyi Wang and Yogeshwar Nath Mishra and Simeng Qiu and Wolfgang Heidrich},
journal = {Optica Imaging Congress 2024 (3D, AOMS, COSI, ISA, pcAOP)},
pages = {CW3B.2},
publisher = {Optica Publishing Group},
title = {Dynamic Fourier Ptychography via Space-Time Optimization},
year = {2024},
url = {https://opg.optica.org/abstract.cfm?URI=COSI-2024-CW3B.2},
}