Local Boundary Regularity of the Szego Projection and Biholomorphic Mappings of Non-Pseudoconvex Domains

Abstract

Abstract. It is shown that the Szegő projection S of a smoothly bounded domain Ω, not necessarily pseudoconvex, satisfies local regularity estimates at certain boundary points, provided that condition R holds for Ω.It is also shown that any biholomorphic mapping f : Ω → D between smoothly bounded domains extends smoothly near such points, provided that a weak regularity assumption holds for D. PreliminariesThroughout, Ω denotes a smoothly bounded domain in C n and r a C ∞ defining function of Ω. The notation W s (Ω), s ∈ R, stands for the Sobolev space of order s. The closure of C ∞ 0 (Ω) in W s (Ω), s > 0, is denoted by W s 0 (Ω) with dual space W −s (Ω). The norm of W −s (Ω) is then u −s = sup{| u, φ |; φ ∈ C ∞ 0 (Ω), φ s = 1}, u ∈ W −s (Ω). Also the dual space (W s (Ω)) * of W s (Ω) is defined, and the norm of u ∈ (W s (Ω)) * , iss is defined to be · s if s ≥ 0, and · * s if s < 0. W s (∂Ω), s ∈ R, denotes the boundary Sobolev space. Any function in W s (∂Ω) is identified with a harmonic function in W s+1/2 (Ω) via the Poisson integral with equivalent norms:where C is a constant independent of u (The letter C in this paper denotes a positive constant which may vary at each of its occurrences.) There is also a local equivalence of these two norms. If ζ 1 , ζ 2 are in C ∞ 0 (C n ) and ζ 2 ≡ 1 near supp ζ 1 , the support of ζ 1 , thenReceived by the editors September 25, 1995 and, in revised form, July 30, 1996 1991 Mathematics Subject Classification. Primary 32H10. where M > 0 is an arbitrary integer and C independent of u.For each integer t ≥ 0, let P t be the orthogonal projection of W t (Ω) to its closed subspace consisting of holomorphic functions. Note that P 0 = P is the usual Bergman projection. If K(w, z) is the Bergman kernel function, P u(z) = u, K(·, z) for u ∈ L 2 (Ω). The Szegő projection S is the orthogonal projection from L 2 (∂Ω) = W 0 (∂Ω) onto the closed subspace consisting of functions whose Poisson integrals are holomorphic in Ω. Similarly, the Szegő projection Su of u is represented by integration against the Szegő kernel S(z, w):where dσ w is the differential surface element on ∂Ω. Definition ([9]).A domain Ω satisfies condition R, if the Bergman projection P of Ω maps C ∞ (Ω) into C ∞ (Ω); and Ω satisfies local condition R at z 0 ∈ ∂Ω, if P maps C ∞ (Ω) to a subspace of holomorphic functions on Ω which are smoothly extendible to the boundary near z 0 .For example, smoothly bounded pseudoconvex domains of finite type in the sense of D'Angelo and also Reinhardt domains satisfy condition R (see [13], [2], and [7]). The latter need not be pseudoconvex.Condition R was introduced by Bell and Ligocka in [9], and has proved to be extremely useful in the study of biholomorphic and proper holomorphic mappings between smoothly bounded domains. Results on regularity of the Bergman projection are often derived from the∂-Neumann theory through Kohn's formula ([18]) which relates them. Now assume that Ω is a smoothly bounded domain in C n , not necessarily pseudoconvex. Definition ([20]).A point z 0 on ...