Cloud tomographic retrieval algorithms. I: Surrogate minimization method

Doicu, Adrian; Doicu, Alexandru; Efremenko, Dmitry; Trautmann, Thomas

doi:10.1016/j.jqsrt.2021.107954

Cited by 8 publications

(23 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They present demonstrations based on the Airborne Multiangle SpectroPolarimetric Imager (AirMSPI) [18], [41]. Doicu et al [42] presents another method for tomographic cloud retrievals, also based on SHDOM. Doicu et al [43] further develop an algorithm based on an adjoint method for gradient computation.…”

Section: A State-of-the-artmentioning

confidence: 99%

Settings for Spaceborne 3-D Scattering Tomography of Liquid-Phase Clouds by the CloudCT Mission

Tzabari

Holodovsky

Shubi

et al. 2022

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

We introduce a comprehensive method for spaceborne 3D volumetric scattering-tomography of cloud microphysics, developed for the CloudCT mission. The retrieved micro-physical properties are the liquid-water-content and effective droplet radius within a cloud. We include a model for a perspective polarization imager, and an assumption of 3D variation of the re. Elements of our work include computed tomography initialization by a parametric horizontally-uniform micro-physical model. This results in smaller errors than the prior art. The mean absolute errors of the retrieved liquid-watercontent and effective-radius are reduced from 62% and 28% to 40% and 9%, respectively. The parameters of this initialization are determined by a grid search of a cost function. Furthermore, we add viewpoints in the cloudbow region, to better sample the polarized scattering phase function. The suggested advances are evaluated by retrieval of a set of clouds generated by large-eddy simulations.

show abstract

Section: A State-of-the-artmentioning

confidence: 99%

Settings for Spaceborne 3-D Scattering Tomography of Liquid-Phase Clouds by the CloudCT Mission

Tzabari

Holodovsky

Shubi

et al. 2022

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

show abstract

“…At the time of writing, there is no publicly available adjoint to a deterministic (i.e., explicit) 3D RTE solver appropriate to the atmospheric context like SHDOM. A forwardadjoint linearization of the SHDOM method has been developed (Doicu and Efremenko, 2019), and an SHDOM solver has been extended so that general adjoints appropriate for tomography can be computed (Doicu et al, 2022b). This forward-adjoint linearization, following the theory of Martin et al (2014), is also technically able to compute the Jacobian matrix of partial derivatives of the forward model.…”

Section: Introductionmentioning

confidence: 99%

“…However, so far, only gradient-based optimization methods have been used, and it is unclear how robust or efficient the approximate linearization will be when combined with optimization methods which make direct use of the Jacobian matrix. Interestingly, the forward-adjoint method of cloud tomography using SHDOM suffered from slow convergence, and the authors only found success in their synthetic tomographic retrievals when utilizing the approximate linearization of Levis et al (2020) and Doicu et al (2022a) in combination with their adjoint method (Doicu et al, 2022b).…”

Section: Introductionmentioning

confidence: 99%

Retrieving 3D distributions of atmospheric particles using Atmospheric Tomography with 3D Radiative Transfer – Part 1: Model description and Jacobian calculation

et al. 2023

View full text Add to dashboard Cite

Abstract. Our global understanding of clouds and aerosols relies on the remote sensing of their optical, microphysical, and macrophysical properties using, in part, scattered solar radiation. These retrievals assume that clouds and aerosols form plane-parallel, homogeneous layers and utilize 1D radiative transfer (RT) models, limiting the detail that can be retrieved about the 3D variability in cloud and aerosol fields and inducing biases in the retrieved properties for highly heterogeneous structures such as cumulus clouds and smoke plumes. To overcome these limitations, we introduce and validate an algorithm for retrieving the 3D optical or microphysical properties of atmospheric particles using multi-angle, multi-pixel radiances and a 3D RT model. The retrieval software, which we have made publicly available, is called Atmospheric Tomography with 3D Radiative Transfer (AT3D). It uses an iterative, local optimization technique to solve a generalized least squares problem and thereby find a best-fitting atmospheric state. The iterative retrieval uses a fast, approximate Jacobian calculation, which we have extended from Levis et al. (2020) to accommodate open and periodic horizontal boundary conditions (BCs) and an improved treatment of non-black surfaces. We validated the accuracy of the approximate Jacobian calculation for derivatives with respect to both the 3D volume extinction coefficient and the parameters controlling the open horizontal boundary conditions across media with a range of optical depths and single-scattering properties and find that it is highly accurate for a majority of cloud and aerosol fields over oceanic surfaces. Relative root mean square errors in the approximate Jacobian for a 3D volume extinction coefficient in media with cloud-like single-scattering properties increase from 2 % to 12 % as the maximum optical depths (MODs) of the medium increase from 0.2 to 100.0 over surfaces with Lambertian albedos <0.2. Over surfaces with albedos of 0.7, these errors increase to 20 %. Errors in the approximate Jacobian for the optimization of open horizontal boundary conditions exceed 50 %, unless the plane-parallel media providing the boundary conditions are optically very thin (∼0.1). We use the theory of linear inverse RT to provide insight into the physical processes that control the cloud tomography problem and identify its limitations, supported by numerical experiments. We show that the Jacobian matrix becomes increasing ill-posed as the optical size of the medium increases and the forward-scattering peak of the phase function decreases. This suggests that tomographic retrievals of clouds will become increasingly difficult as clouds become optically thicker. Retrievals of asymptotically thick clouds will likely require other sources of information to be successful. In Loveridge et al. (2023a; hereafter Part 2), we examine how the accuracy of the retrieved 3D volume extinction coefficient varies as the optical size of the target medium increases using synthetic data. We do this to explore how the increasing error in the approximate Jacobian and the increasingly ill-posed nature of the inversion in the optically thick limit affect the retrieval. We also assess the accuracy of retrieved optical depths and compare them to retrievals using 1D radiative transfer.

show abstract

“…So far, only local optimization methods have been employed for physics-based cloud tomography (Martin and Hasekamp, 2018;Levis et al, 2020;Doicu et al, 2022b). The proposed BFGS algorithm performs well compared to other local optimization techniques (Doicu et al, 2022a). Global optimization methods such as ensemble-based particle filters (van Leeuwen et al, 2019) have also been proposed for tomography in other fields (Raveendran et al, 2011).…”

Section: Introductionmentioning

confidence: 99%

Retrieving 3D distributions of atmospheric particles using Atmospheric Tomography with 3D Radiative Transfer – Part 2: local optimization

Loveridge

Levis

Girolamo

et al. 2023

Preprint

View full text Add to dashboard Cite

Abstract. Our global understanding of clouds and aerosols relies on the remote sensing of their optical, microphysical, and macrophysical properties using, in part, scattered solar radiation. Current retrievals assume clouds and aerosols form plane-parallel, homogeneous layers and utilize 1D radiative transfer (RT) models. These assumptions limit the detail that can be retrieved about the 3D variability of cloud and aerosol fields and induce biases in the retrieved properties for highly heterogeneous structures such as cumulus clouds and smoke plumes. In Part 1 of this two-part study, we validated a tomographic method that utilizes multi-angle passive imagery to retrieve 3D distributions of species using 3D RT to overcome these issues. That validation characterized the uncertainty in the approximate Jacobian used in the tomographic retrieval over a wide range of atmospheric and surface conditions for several horizontal boundary conditions. Here in Part 2, we test the algorithm’s effectiveness on synthetic data to test whether retrieval accuracy is limited by the use of the approximate Jacobian. We retrieve 3D distributions of volume extinction coefficient (σ3D) at 40 m resolution from synthetic multi-angle, mono-spectral imagery at 35 m resolution derived from stochastically-generated ‘cumuliform’ clouds in (1 km)3 domains. The retrievals are idealized in that we neglect forward modelling and instrumental errors with the exception of radiometric noise; thus reported retrieval errors are lower bounds. σ3D is retrieved with, on average, a Relative Root Mean Square Error (RRMSE) < 20 % and bias < 0.1 % for clouds with Maximum Optical Depth (MOD) < 17, and the RRMSE of the radiances is < 0.5 %, indicating very high accuracy in shallow cumulus conditions. As the MOD of the clouds increases to 80, the RRMSE and biases in σ3D worsen to 60 % and −35 %, respectively, and the RRMSE of the radiances reaches 16 %, indicating incomplete convergence. This is expected from the increasing ill-conditioning of the inverse problem with decreasing mean-free-path predicted by RT theory and discussed in detail in Part 1. We tested retrievals that use a forward model that is better conditioned but less accurate due to more aggressive delta-M scaling. This reduces the radiance RRMSE to 9 % and the bias in σ3D to −8 % in clouds with MOD ~80, with no improvement in the RRMSE of σ3D. This illustrates a significant sensitivity of the retrieval to the numerical configuration of the RT model which, at least in our circumstances, improves the retrieval accuracy. All of these ensemble-averaged results are robust to the inclusion of radiometric noise during the retrieval. However, individual realizations can have large deviations of up to 18 % in the mean extinction in clouds with MOD ~80, which indicates large uncertainties in the retrievals in the optically thick limit. Using the better conditioned forward model tomography can also accurately infer optical depths (OD) in conditions spanning the majority of oceanic, cumulus fields (MOD < 80) as the retrieval provides OD with bias and RRMSE better than −8 % and 36 %, respectively. This is a significant improvement over retrievals using 1D RT, which have OD biases between −30 % and −23 % and RRMSE between 29 % and 80 % for the clouds used here. Prior information or other sources of information will be required to improve the RRMSE of σ3D in the optically thick limit, where the RRMSE is shown to have strong spatial structure that varies with the solar and viewing geometry.

show abstract

Cloud tomographic retrieval algorithms. I: Surrogate minimization method

Cited by 8 publications

References 21 publications

Settings for Spaceborne 3-D Scattering Tomography of Liquid-Phase Clouds by the CloudCT Mission

Settings for Spaceborne 3-D Scattering Tomography of Liquid-Phase Clouds by the CloudCT Mission

Retrieving 3D distributions of atmospheric particles using Atmospheric Tomography with 3D Radiative Transfer – Part 1: Model description and Jacobian calculation

Retrieving 3D distributions of atmospheric particles using Atmospheric Tomography with 3D Radiative Transfer – Part 2: local optimization

Contact Info

Product

Resources

About