A simple variable transformation that consists of two joined straight-line segments per grating period is proposed for the parameterized coordinate transformation method (the C method). With this bilinear parameterization, the C method can produce convergent numerical results for gratings of deep and smooth profiles with a groove depth-to-period ratio as high as 10, which to date has been far out of reach of the C method. The danger of getting divergent results due to inadvertently using an overly large truncation number is also practically eliminated.

The translational coordinate transformation method (the C method) in grating theory is studied numerically and analytically. We first study the convergence characteristics of the C method by numerical computations in high floating-point data precisions. Guided by insights gained from this numerical study we analytically studied condition numbers of the most important eigenvalues of the eigenvalue problem of the C method. Asymptotic estimates of condition numbers of these eigenvalues and estimates of convergence rate of the error in satisfying the Helmholtz equation by the eigenvectors are derived. These theoretical results explain well many observed numerical phenomena of the C method. Using the first-order perturbation theory of simple eigenvalues we analyze the effects of round-off errors on eigenvalue distribution and condition numbers. This leads to an extremely simple perturbative preconditioning technique that significantly improves the numerical stability of the C method with as little as just one line of code modification. The performance of the perturbatively preconditioned C method is not inferior to the C method preconditioned by the multilinear parameterization technique. We recommend it as the preferred method for modeling deep and smooth gratings.

We recently found that the coordinate transformation method (the C method) equipped with well-established recursive algorithms for solving the system of linear equations is numerically instable when it is applied to thinly coated gratings. The origin of this new kind of numerical instability is not the exponential dependence of the field in the coated layers but the ill condition of the eigenvector matrix of the C method when the truncation number is high. Two simple and effective methods to circumvent the new instability are recommended. We also found that the popular recursive matrix algorithms have different (poor) immunities to the new instability, and they all perform inferiorly to the full matrix (nonrecursive) algorithm.

We present two piecewise linear parameterization schemes for the parameterized coordinate transformation method (the C method) to enable it to model gratings of deep and smooth grooves. This work generalizes and elaborates our previous work [Opt. Lett. 39(23), 6644 (2014)]. The previous bilinear transformation is replaced with a general multi-linear transformation. This gives us flexibility to handle more general grating profiles while retaining the simplicity of the linear transformation. We give some general, simple, and empirical rules on composing the piecewise linear transformation function. Both enlarged convergence range and increased groove depth-to-period ratio, which can be at least 10, are achieved with the parameterized C method for a wide class of smooth grating profiles.

No abstract