Proceedings of Symposia in Pure Mathematics, Vol. 52 (1991), Part 3, 219–231

A study of the Bergman projection in certain Hartogs domains

Christer O. Kiselman

Contents:

1. Introduction

2. Worm domains

3. Functions which are locally independent of the second variable

4. The Bergman projection in a disk

5. A domain where the Bergman projection does not preserve smoothness 6. A smoothly bounded domain in a manifold

References

Resumo: Studo de la projekcio de Bergman en kelkaj Hartogs-aj regionoj

Oni montras ke la Bergman-a projekcio ne konservas glatecon de funkcioj en kelkaj pseŭdokonveksaj malfermitaj aroj en la spaco de du kompleksaj variabloj.

Abstract: We show that the Bergman projection does not preserve smoothness of functions in some pseudoconvex domains in the space of two complex variables.

1. Introduction

The purpose of this paper is to present two examples of domains where the Bergman projection does not preserve smoothness.

In the space L^{2}(Ω) of square-integrable functions in an open set Ω ⊂ C^{n}, the holomorphic functions form a closed subspace O^{2}(Ω) = L^{2}(Ω)∩O(Ω), the Bergman space. The orthogonal projection onto that subspace is the Bergman projection P : L^{2}(Ω) → O^{2}(Ω). If Ω is bounded, the space C^{∞}(Ω̄) of functions which are smooth up to the boundary is a subset of L^{2}(Ω). The question we study is whether P preserves smoothness in the sense that P (C^{∞}(Ω̄)) is contained in C^{∞}(Ω̄).

A Hartogs domain in the space C^{2} of two complex variables is a domain which contains along with (z, w) also every point (z, w′) with |w′| = |w|. It is said to be a complete Hartogs domain if it contains with (z, w) also (z, w′) for all w′ with |w′| ⩽ |w|. Barrett [1984] has found a smoothly bounded Hartogs domain in C^{2} where the Bergman projection does not preserve smoothness. His domain is not pseudoconvex. It is interesting to compare this with the result of Boas & Straube [1989] which shows that in a complete Hartogs domain (whether pseudoconvex or not) the Bergman projection maps C^{∞}(Ω̄) into itself.

Here we will show that in a suitable pseudoconvex Hartogs domain in C^{2}, whose boundary is not smooth, the Bergman projection does not preserve C^{∞}(Ω̄). This gives rise to a smoothly bounded pseudoconvex open subset of a two-dimensional manifold C×((C\{0})/∼) where the Bergman projection does not preserve smoothness.

Theorem 1. Let

Ω = {(z, w) ∈ C^{2}; a < log |w| < b & |z − |w|^{i}| < 1},

where b − a = mπ for some m = 1, 2, 3, ... . It is a bounded pseudoconvex domain and its boundary is C^{∞} except at the points which satisfy |w| = e^{a}, |z − e^{ia}| = 1 or |w| = e^{b}, |z − e^{ib}| = 1. Then there is a function f ∈ C^{∞}(Ω̄) such that its Bergman projection P f is not Hölder continuous in Ω̄. Now let θ be a positive number and let X_{θ} denote the compact complex manifold obtained from C \ {0} by identifying w and e^{θ}w. If θ is an integer multiple of 2π, then

Y = {(z, w) ∈ C × X_{θ}; |z − |w|^{i}| < 1}

is a pseudoconvex domain in C × X_{θ} with real-analytic boundary. We can define in a natural way a Bergman space O^{2}(Y) consisting of differential forms of type (2, 0) and a Bergman projection P : L^{2}(Y) → O^{2}(Y) (see section 6).

Theorem 2. Let Y be the smoothly bounded pseudoconvex domain just defined, with θ = 2πm for some m = 2, 3, 4, ... . Then the Bergman projection does not map the forms with coefficients in C^{∞}(Ȳ) into the space of continuous forms on Ȳ. Domains with similar properties were studied by Barrett [1986].

The structure of the proof of Theorem 1 is as follows. On a subspace C^{+}(Ω) of L^{2}(Ω) which contains all Hölder-continuous functions in Ω̄ we construct a linear functional T whose values are obtained as holomorphic extensions of inner products ⟨f, g_{α}⟩ with certain elements g_{α} of the Bergman space. More precisely, for a fixed f ∈ C^{+}(Ω) we define a holomorphic function Φ(α) = ⟨f, g_{α}⟩ of a complex variable α in the half-plane Re α > −1. Then T (f) is the point value T (f) = Φ(−2) of the extension. The functional T has the following properties.

(a) If both f and P f belong to C^{+}(Ω), then T (P f) = T (f), for f −P f is orthogonal to O^{2}(Ω), in particular to the g_{α}.

(b) T (f) = 0 if f is in C^{+}(Ω) and holomorphic.

(c) T is not identically zero on C^{+}(Ω); more precisely, C^{+}(Ω) contains C^{∞}(Ω̄) and T is not zero on the latter space. To finish the proof we only have to take an f ∈ C^{∞}(Ω̄) with T (f) ≠ 0. If

P f did belong to C^{+}(Ω), then it would follow from (a) that T (P f) = T (f) ≠ 0, contradicting (b). For details see section 5.

The logical setup of the proof of Theorem 2 is similar, but the functional we use is different. See section 6.

I am grateful to John Erik Fornæss for comments on an early version of this paper.

2. Worm domains

Diederich & Fornæss [1977] defined a one-parameter family of pseudoconvex domains which have become known as worm domains. Each of them is a domain Ω in the space of two complex variables (z, w) whose intersection with any complex subspace w = constant is a disk in the z-plane with center at the point |w|^{i} = exp(i log |w|) and radius R(log |w|) ⩽ 1. If R(log |w|) = 1, then the point (0, w) is on the boundary of Ω. Let us agree to call any domain of the form

Ω = {(z, w) ∈ C^{2}; |z − |w|^{i}| < R(log |w|)}

a worm domain. In particular they are Hartogs domains and can be easily visualized in the space of three real variables (Re z, Im z, log |w|). We shall give a condition on the radius R which implies pseudoconvexity. It will be convenient to consider ψ = 1 − R^{2}. We allow ψ < 0, i. e., R > 1. Where ψ > 1 there are no corresponding points in Ω.

Let us say that a function ψ defined on some interval of R is trigonometrically convex if ψ′′ + ψ ⩾ 0 in the distribution sense. This amounts to ψ(arg ζ)|ζ| being convex (or, equivalently, subharmonic) in some sector in the complex plane. The easiest examples are the convex nonnegative functions. Other examples are a sin and a sin^{+} = a max(sin, 0), a > 0.

The following result is due to Diederich & Fornæss [1977], pages 277–279, for a particular choice of the function ψ.

Proposition 2.1. Let −∞ ⩽ a < b ⩽ +∞ and let ψ be a trigonometrically convex function on the open interval ]a, b[. Then

(2.1)  Ω = {(z, w) ∈ C^{2}; a < log |w| < b & |z − |w|^{i}|^{2} + ψ(log |w|) < 1}

is pseudoconvex. If ψ is of class C^{k} for some k = 1, 2, 3, ..., ∞, then ∂Ω is of class C^{k} except possibly where ψ = 1 or |w| = e^{a} or |w| = e^{b}; if in addition ψ(a+) > 1, ψ(b−) > 1 and ψ′ ≠ 0 when ψ = 1, then ∂Ω is of class C^{k} everywhere.

Proof. Locally we can introduce ζ = w^{−i} as a new variable; then log |w| = − arg ζ and arg w = log |ζ|. The condition for a point (z, w) with a < log |w| < b to belong to Ω, in addition to the obvious a < − arg ζ < b, becomes

|z|^{2} − 2 Re(ze^{i arg ζ}) + ψ(− arg ζ) < 0.

We can multiply this by |ζ| to get the equivalent condition

|z|^{2}|ζ| − 2 Re(zζ) + ψ(− arg ζ)|ζ| < 0.

Here all terms are plurisubharmonic in (z, ζ): the first is the exponential of the plurisubharmonic function 2 log |z| + log |ζ|, the second is the real part of a holomorphic function, and the third is convex in ζ by hypothesis. Therefore the set where this function is negative is pseudoconvex.

If ψ(a+) > 1 and ψ(b−) > 1 there can be a question about the smoothness of the boundary only where the radius of the disk in the z-plane is zero, i. e., where ψ = 1. Consider the domain

Ω′ = {(z, w) ∈ C^{2}; a < log |w| < b & |z − 1|^{2} + ψ(log |w|) < 1}

which is diffeomorphic to Ω under the diffeomorphism (z, w) ↦ (z|w|^{−i}, w) (defined in a neighborhood of Ω̄). If ψ satisfies the condition mentioned at the points where it takes the value 1, then the boundary of Ω′ near the points where the radius is zero is given by the equation ψ(log |w|) = 1 − |z − 1|^{2}. We can write this as log |w| = ψ^{−1}(1 − |z − 1|^{2}), where z ↦ 1 − |z − 1|^{2} is C^{∞}, and where the inverse ψ^{−1} is as smooth as ψ. This proves the proposition.

The simplest choice is a = −∞, b = +∞, R = 1 identically; this defines an unbounded pseudoconvex domain Ω_{0} whose boundary is C^{∞} except where w = 0. Another choice is −∞ < a < b < +∞, R = 1; then Ω is bounded and its boundary is smooth except where |w| = e^{a}, |z − e^{ia}| = 1 or |w| = e^{b}, |z − e^{ib}| = 1. The boundary of these domains contains a piece of the analytic manifold z = 0.

It is easy to define smoothly bounded domains using Proposition 2.1: just take a C^{∞} convex function ψ on [a, b] which is zero on some interval [a_{0}, b_{0}], positive outside this interval and satisfies ψ(a) > 1, ψ(b) > 1. Since ψ is nonnegative and convex, it is trigonometrically convex, and Ω is contained in the set where ψ < 1. There exist two points a_{1} < b_{1} such that ψ(a_{1}) = ψ(b_{1}) = 1; obviously ψ′(a_{1}) < 0, ψ′(b_{1}) > 0. Therefore ∂Ω is C^{∞}.

We now ask whether the domains in Proposition 2.1 can be defined by a global plurisubharmonic function. Diederich & Fornæss [1977] proved (pages 284–285) that their worm domains do not possess a plurisubharmonic defining function of class C^{3}. We shall prove a somewhat stronger result here.

Proposition 2.2. Let Ω be as in Proposition 2.1 with ψ ⩾ 0 everywhere and ψ = 0 (i. e., R = 1) on some interval [a_{0}, b_{0}] of positive length. Denote the distance to the boundary in Ω by d_{Ω}. Then there is no function f ∈ PSH(Ω) satisfying −Ad_{Ω} ⩽ f ⩽ −Bd_{Ω} in Ω with positive constants A and B. In particular there cannot exist a global plurisubharmonic defining function for Ω which is Lipschitz continuous. A little more generally, if ε > π/(b_{0} −a_{0} +π), there cannot exist a function f ∈ PSH(Ω) satisfying −Ad_{Ω}^{ε} ⩽ f ⩽ −Bd_{Ω}^{ε} in Ω with positive A and B.

Proof. Suppose there is a plurisubharmonic function f with −Ad_{Ω}^{ε} ⩽ f ⩽ −Bd_{Ω}^{ε}. Define first

f_{0}(z, w) = sup_{|w′|=|w|} f(z, w′), (z, w) ∈ Ω.

Then f_{0} is a function of (z, |w|), it is plurisubharmonic in Ω and satisfies the same inequality as f there, for d_{Ω}(z, w) = d_{Ω}(z, w′) if |w| = |w′|. Next define

F (z) = inf_{w} f_{0}(z, w).

In view of the minimum principle for plurisubharmonic functions (Kiselman [1978], Theorem 2.2), F is subharmonic on some Riemann surface in z. We may suppose without loss of generality that a_{0} < 0 < b_{0}, so that this Riemann surface contains the half-disk

U = {z ∈ C; |z| < r & Re z > 0}

for some positive r. If −Ad_{Ω} ⩽ f ⩽ −Bd_{Ω}, then F satisfies −A_{1}|z| ⩽ F (z) ⩽ −B_{1}|z| in U for some new positive constants A_{1} and B_{1}, and F has a continuous extension to the closure Ū. We introduce an auxiliary function

G(z) = |z|(1 − \frac{1}{2} log \frac{|z|}{r} cos arg z).

We note that G is subharmonic in U, has a continuous extension to the closure of U and coincides with |z| on the boundary of U. Consider F + B_{1}G. This function is ⩽ 0 on the boundary of U. But at the point z = r exp(−2A_{1}/B_{1}) ∈ U it assumes a positive value: this contradicts the maximum principle for subharmonic functions and completes the proof in the case ε = 1.

For the general case we define F_{ε}(z) = F (z^{1/ε}) in U for a sufficiently small positive r. This is possible if arg i^{1/ε} < b_{0} + π/2 and arg(−i)^{1/ε} > a_{0} − π/2. It is however no restriction to assume that b_{0} = −a_{0}; then both conditions become π/ε < b_{0} − a_{0} + π. The function F_{ε} has a continuous extension to Ū and satisfies −A_{1}|z| ⩽ F_{ε}(z) ⩽ −B_{1}|z|. Thus the same argument applies, completing the proof of Proposition 2.2.

It is interesting to note in this context the result due to Bonami & Charpentier [1988]: the Bergman projection preserves the Sobolev class W^{1/2}(Ω) in a domain with boundary of class C^{3} and admitting a global plurisubharmonic defining function.

3. Functions which are locally independent of the second variable

In the sets Ω as defined in the last section the functions which are locally independent of the second variable w play an important role.

Proposition 3.1. Let Ω be a Hartogs domain in C^{2}. If f ∈ L^{2}(Ω) is of the form f(z, w) = w^{k}F (z, |w|) for some k ∈ Z, then w^{−k}P f is locally independent of w.

Proof. If a is a complex number of modulus one, the mapping f ↦ f ∘ T_{a} defined by T_{a}(z, w) = (z, aw) is an isometry T_{a}^{∗} of L^{2}(Ω) which moreover preserves the subspace O^{2}(Ω). This implies that T_{a}^{∗} and P commute. Now if f(z, w)/w^{k} is a function of (z, |w|) (for w ≠ 0), we have T_{a}^{∗}f = a^{k}f for all numbers a of modulus one, so that T_{a}^{∗}P f = P T_{a}^{∗}f = P (a^{k}f) = a^{k}P f. Therefore H(z, w) = w^{−k}P f(z, w) satisfies T_{a}^{∗}H = H: it is invariant under rotation in the w-space and so must be a function of (z, |w|). Since H is holomorphic, this implies that it is locally independent of w.

Proposition 3.2. Let Ω be defined as in Proposition 2.1 with a nonnegative ψ such that ψ^{−1}(0) is an interval. Assume f ∈ C(Ω) is locally independent of w and lim f(z, w) = f_{0}(w) exists as z → 0, (z, w) ∈ Ω, for every w such that (0, w) ∈ ∂Ω. Then f_{0} is constant.

Proof. In particular f_{0} is equal to the limit

f_{0}(w) = lim_{t→0} f(tz, w)

where t > 0 and z is chosen so that (tz, w) is in Ω for all t with 0 < t < ε for some positive ε. The same z can serve for all w near the original point. Therefore f_{0} is locally constant. Now the set of w we are considering is connected: it is the annulus e^{a_{0}} ⩽ |w| ⩽ e^{b_{0}} where [a_{0}, b_{0}] = ψ^{−1}(0). So locally constant is the same thing as being constant.

It will be important for us to have a supply of functions which are holomorphic and locally independent of w. Let Ω_{0} be the set defined by (2.1) with the special choices a = −∞, b = +∞, and ψ = 0. We define a function in Ω_{0} which we call simply log z, by first defining it for w = 1 in the disk

D_{1} = {z ∈ C; |z − 1| < 1}

as the branch of the logarithm with log 1 = 0. Then we continue it analytically from D_{1} ×{1} ⊂ Ω_{0} to all of Ω_{0}. As in Diederich & Fornæss [1977], § 2, we can now define, for any complex number α, a function

g_{α}(z, w) = exp(α log z)

of (z, w) which we may also denote by z^{α}, at least if we keep in mind that it is not globally a function of z when α ∈ C \ Z.

4. The Bergman projection in a disk

Here we shall recall some formulas for the Bergman projection in a disk. In the unit disk D = {z ∈ C; |z| < 1} we have an explicit expression for the Bergman projection B : L^{2}(D) → O^{2}(D):

(4.1)  Bf(a) = ∫_{D} f(z)\frac{1}{π(1 − z̄a)^{2}} dλ(z),  a ∈ D;

see e. g. Krantz [1982: 50]. Here λ denotes Lebesgue measure in C. We change variables in (4.1) to get the projection in the disk D_{r} = {z ∈ C; |z − 1| < r} and write B : L^{2}(D_{r}) → O^{2}(D_{r}) for the Bergman projection there also. This gives

(4.2)  Bf(a) = ∫_{D_{r}} f(z)\overline{φ_{a}(z)} dλ(z),  a ∈ D_{r},

where

φ_{a}(z) = \frac{(1 − a^{∗})^{2}}{πr^{2}(z − a^{∗})^{2}} , a, z ∈ D_{r},

and a^{∗} is the point defined by reflection in ∂D_{r}, in other words a^{∗} is determined by (ā − 1)(a^{∗} − 1) = r^{2}. If 0 < r < 1, we can in particular choose a^{∗} = 0, a = 1 − r^{2}:

(4.3)  ∫_{D_{r}} \frac{f(z)}{z̄^{2}} dλ(z) = πr^{2}Bf(1 − r^{2}),  0 < r < 1.

In a suitable sense (4.3) holds also for r = 1 if f is Hölder continuous in D̄_{1}. The interpretation of the integral is then elementary if f(0) = 0, and one can prove that

∫_{D_{1}} \frac{f(z)}{z̄^{2}}dλ(z) = πBf(0)

by passing to the limit in (4.2). Here we let Bf denote also the continuous extension to the closure of D_{1}. Next we write f(z) = f(0)+(f(z)−f(0)) and consider the case when f is identically one. Then we note that, in view of the mean value property for antiholomorphic functions,

∫_{D_{r}} z̄^{α} dλ(z) = πr^{2}, 0 < r ⩽ 1, Re α > −2.

We simply extend this function of (r, α) by continuity from Re α > −2 to Re α ⩾ −2. This is the interpretation of (4.3) for r = 1 which we shall use.

Note that we do not define (4.3) for r = 1 as a principal value by cutting out a disk around the origin: that will give a different value. In fact, it can be proved that

lim_{ε→0} ∫_{z∈D_{1},|z|>ε} \frac{1}{z̄^{2}} dλ(z) = \frac{π}{2} .

5. A domain where the Bergman projection does not preserve smoothness

Every holomorphic function in a Hartogs domain Ω has a Laurent expansion

h(z, w) = ∑_{j∈Z} h_{j}(z, |w|)w^{j},

where h_{j} ∈ O(Ω). It turns out that in the worm domains (2.1) the functions which are homogeneous of degree −1 exhibit a resonance phenomenon which we will exploit. (It will give rise to property (b) of the introduction, and is made explicit in formula (5.4) below.) We shall therefore calculate the scalar product ⟨f/w, g/w⟩ where f and g are two functions in L^{2}(Ω) satisfying f(z, w) = f(z, |w|), g(z, w) = g(z, |w|), but we shall not suppose that they are holomorphic. Introducing polar coordinates for w with |w| = r = e^{t} we get

⟨f/w, g/w⟩ = ∫∫_{Ω} \frac{f(z, w)}{w} \frac{\overline{g(z, w)}}{w̄}dλ(z)dλ(w) =

= 2π ∫∫ f(z, r)\overline{g(z, r)}dλ(z)dr/r = 2π ∫∫_{|z−e^{it}|<R(t)} f(z, e^{t})\overline{g(z, e^{t})}dλ(z)dt.

We now take z′ = ze^{−it} as a new variable: the transformation (z, w) ↦ (z′, w) preserves volumes and we get

⟨f/w, g/w⟩ = 2π ∫∫_{|z′−1|<R(t)} f(e^{it}z′, e^{t})\overline{g(e^{it}z′, e^{t})}dλ(z′)dt.

If Ω is bounded and defined by a nonnegative function ψ, and if Re α > −1, we can in particular choose g(z, w) = zᾱ and define

(5.1)  \begin{aligned}T_{α}(f/w) & = ⟨f/w, g/w⟩ = 2π ∫∫_{|z−1|<R(t)} f(e^{it}z, e^{t})\overline{g(e^{it}z, e^{t})}dλ(z)dt = \\ & = 2π ∫∫_{|z−1|<R(t)} f(e^{it}z, e^{t})e^{−iαt}z̄^{α}dλ(z)dt. \end{aligned}

Moreover

(5.2)  T_{α}(f/w) = T_{α}(P (f/w))

for these α, since (z, w) ↦ zᾱ/w belongs to O^{2}(Ω) and f/w − P (f/w) is orthogonal to that space. Note that, by Proposition 3.1, the function P (f/w) is of the form h(z, |w|)/w so that the scalar product ⟨P (f/w), g/w⟩ is given by the same formulas.

The equation (5.2) will of course be preserved under analytic continuation in the parameter α. For example, if both f and P (f/w) happen to be bounded, we can extend the equation to the half-plane Re α > −2.

Now if f is measurable and satisfies an estimate |f(z, w)| ⩽ C|z|^{ε} with ε > 0, then T_{α}(f/w) is well defined by (5.1) for Re α > −2 − ε.

If f is continuous on the closure of Ω and independent of z, we may extend f to all of C^{2} and write f(0, w) for the value of f at any of the points (z, w) ∈ Ω̄ (even though the corresponding point (0, w) need not belong to Ω̄). In this case we can easily extend T_{α}(f/w):

T_{α}(f/w) = 2π ∫∫_{|z−1|<R(t)} f(0, e^{t})e^{−iαt}z̄^{α}dλ(z)dt = 2π^{2} ∫ R(t)^{2}f(0, e^{t})e^{−iαt}dt,

for in view of the mean value property for antiholomorphic functions, the integral with respect to z is equal to the area πR(t)^{2} times the value at the center of the disk. The last integral has a sense for all α ∈ C.

If f is Hölder continuous with exponent ε in Ω̄ we can extend T_{α}(f/w) to all α satisfying Re α > −2 − ε by combining these two ideas: we shall assume that f is defined in all of C^{2} and write f(z, w) = f(0, w) + (f(z, w) − f(0, w)). We now apply the functional T_{α} separately to the two terms. We see that if Re α > −2, then T_{α}(f/w) can be written as

2π^{2} ∫ R(t)^{2}f(0, e^{t})e^{−iαt}dt + 2π ∫∫ (f(e^{it}z, e^{t}) − f(0, e^{t}))e^{−iαt}z̄^{α}dλ(z)dt.

Now the first term has a sense for all α, and the second is defined as an integral for all α in the half-plane Re α > −2 − ε. The extension so obtained seemingly depends on the values of f(0, w) also for w outside the annuli where f(0, w) is defined originally (these annuli are defined by the condition that log |w| belong to ψ^{−1}(0) which is a union of intervals), but it actually does not, since T_{α}(f/w) is a function of f restricted to Ω for α satisfying Re α > −2, and then this must be true also of its holomorphic extension.

Of course we do not need Hölder continuity everywhere in Ω̄, so we can summarize the discussion in the following definition:

Definition 5.1. For a worm domain Ω with R ⩽ 1 we let C^{+}(Ω) denote the space of all bounded measurable functions f defined in Ω which satisfy an estimate

|f(z, w) − f_{0}(w)| ⩽ C|z|^{ε}, (z, w) ∈ Ω,

for a suitable measurable function f_{0} and some constants C and ε > 0. We define

T_{−2}(f/w) = lim_{\begin{matrix}α→−2 \\ Re α>−2 \end{matrix}} T_{α}(M(f)/w), f ∈ C^{+}(Ω),

where T_{α} is given by (5.1) for Re α > −2, and where M(f) denotes the mean value of f(z, tw) over the circle |t| = 1. The function f_{0} can be thought of as an extension of w ↦ f(0, w); it is not unique since (z, w) ∈ Ω̄, R(log |w|) < 1 implies (0, w) ∉ Ω̄.

We can now use the formulas for the Bergman projection in a disk (see section 4) to express T_{−2}. It will be enough to consider functions f ∈ C^{+}(Ω) which are are functions of (z, |w|) so that M(f) = f. If such an f is holomorphic in z for fixed w, we get writing I for the open set where R > 0, ψ < 1, and J = ψ^{−1}(0) for the set where R = 1, ψ = 0,

(5.3)  \begin{aligned}T_{−2}(f/w) & = 2π^{2} ∫ R(t)^{2}f(e^{it}(1 − R(t)^{2}), e^{t})e^{2it}dt = \\ & = 2π^{2} ∫_{J} f(0, e^{t})e^{2it}dt + 2π^{2} ∫_{I\J} R(t)^{2}f(ψ(t)e^{it}, e^{t})e^{2it}dt. \end{aligned}

These formulas clearly use only values of f in Ω̄. It is by no means obvious how to estimate the integral over I \ J. However, in the special case when R = 1 in the whole interval ]a, b[ this term vanishes. Then we obtain the following result, which implies Theorem 1 of the introduction:

Theorem 5.2. Let

Ω = {(z, w) ∈ C^{2}; a < log |w| < b & |z − |w|^{i}| < 1},

where b − a = mπ for some m = 1, 2, 3, ... . Let f ∈ C^{+}(Ω) be holomorphic in z for fixed w and satisfy f(z, w) = f(z, |w|), ∫ f(0, e^{t})e^{2it}dt ≠ 0. Then P (f/w), the Bergman projection of f/w, is not in C^{+}(Ω). We can easily find C^{∞} functions f which satisfy the hypotheses of the theorem; for instance f(z, w) = g(log |w|), where g ∈ C^{∞}(R) with support in [a, b] and Fourier transform ĝ satisfying ĝ(−2) ≠ 0. Proof. In view of Proposition 3.1, P (f/w) = h/w, where h is a holomorphic function of (z, |w|). If h satisfied an estimate |h(z, w) − h(0, w)| ⩽ C|z|^{ε}, we would get T_{−2}(h/w) = T_{−2}(f/w) ≠ 0 from (5.2). On the other hand, (5.3) yields

(5.4)  T_{−2}(h/w) = 2π^{2} ∫_{J} h(0, e^{t})e^{2it}dt = 2π^{2}h(0, e^{s}) ∫_{J} e^{2it} dt = 0,

where we write h(0, e^{s}) for the common value of h(0, w) for all w such that (0, w) ∈ ∂Ω; see Proposition 3.2. The last equality follows simply because J = ]a, b[ = ]a, a + mπ[ so that

∫_{J} e^{2it} dt = ∫_{a}^{a+mπ} e^{2it} dt = 0.

This contradiction proves the theorem.

6. A smoothly bounded domain in a manifold

Let θ be a positive number and let X_{θ} denote the compact complex manifold obtained from C \ {0} by identifying w and e^{θ}w. The differential form dw/w is well defined on X_{θ}. We shall let q : C × (C \ {0}) → C × X_{θ} denote the canonical projection which associates with (z, w) its equivalence class in C × X_{θ}.

Consider the domain

Ω = {(z, w) ∈ C × (C \ {0}); |z − |w|^{i}| < 1},

which is pseudoconvex in view of Proposition 2.1, and its subset

Ω_{θ} = {(z, w) ∈ Ω; 0 < log |w| < θ}.

If θ = 2πm is an integer multiple of 2π, the condition on (z, w) to belong to Ω does not change if we replace w by e^{θ}w, for in that case |e^{θ}w|^{i} = e^{2πmi}|w|^{i} = |w|^{i}. Therefore

(6.1)  Y = q(Ω) = {q(z, w) ∈ C × X_{θ}; |z − |w|^{i}| < 1}

is a pseudoconvex domain in C×X_{θ} with real-analytic boundary. We have q(Ω̄_{θ}) = Ȳ. From now on we shall assume θ = 2πm.

We let L^{2}(Y) denote the space of all differential forms of bidegree (2, 0) with a representation

F = f(z, w) dz ∧ dw/w

where f is square integrable on bounded subsets of Ω and satisfies f(z, e^{θ}w) = f(z, w). The scalar product is

⟨F, G⟩ = ∫∫_{Ω_{θ}} f(z, w) dz ∧ dw/w ∧ \overline{g(z, w) dz ∧ dw/w}.

The Bergman space O^{2}(Y) is the subspace of L^{2}(Y) with f holomorphic. It is contained in a natural way in the Bergman space O^{2}(Ω_{θ}): an element of the latter belongs to the former if it has an extension to all of Ω which is periodic in the sense that f(z, e^{θ}w) = f(z, w). Our result that T_{−2}(f/w) = 0 for all holomorphic Hölder-continuous f in Ω̄_{θ} is therefore true also for Y.

The function g_{α}(z, w) = exp(α log z) (see the end of section 3) is well defined in Y if αθ/2π is an integer. For instance, if θ = 4π, then \sqrt{z} is well defined in Y, although it is not globally a function of z, and both \sqrt{z} dz ∧ dw/w and 1/\sqrt{z} dz ∧ dw/w are elements of the Bergman space. But there seems to be no reason why the equation T_{−2}(P f) = T_{−2}(f), which was crucial in the proof of Theorem 5.2, should hold, for this was a consequence of the fact that T_{−2} appeared as a limit of scalar products with functions zᾱ, and such functions are defined on the manifold X_{θ} = X_{2πm} only if α is a rational number of the form α = k/m with k ∈ Z. (Moreover the corresponding form belongs to the Bergman space only if k > −m.) This is the reason why we have not been able to use the functional T_{−2} in the present proof. We shall therefore construct another functional S as follows.

Definition 6.1. Let γ denote the curve [0, θ] ∋ t ↦ (e^{it}, e^{t}) ∈ Ω̄_{θ} which traces the centers of the disks in the z-plane. Then q ∘ γ is a closed curve in Y. We let N(Y) denote the set of all functions f ∈ C(Y) which do not vanish on q ∘ γ. We define a functional S by the change in argument of the number f(q(z, w)) along γ:

S(f) = \frac{1}{2πi} var_{γ} arg f(q(z, w)) = \frac{1}{2πi} var_{q∘γ} arg f, f ∈ N(Y).

We list a few of the properties of S:

Lemma 6.2. If f ∈ N(Y) ∩ O(Y) ∩ C(Ȳ) is a function of (z, |w|), then S(f) ∈ N.

Proof. We define three segments γ_{0}, γ_{1}, γ_{2} as follows:

γ_{0} : [0, θ] ∋ t ↦ (0, e^{θ−t}),  γ_{1} : [0, 1] ∋ t ↦ (t, 1),  γ_{2} : [0, 1] ∋ t ↦ (1 − t, e^{θ}).

They are all contained in the boundary of Ω_{θ}. Let Γ be the closed curve in Ω̄_{θ} consisting of γ_{1} + γ + γ_{2} + γ_{0}. Now clearly

var_{q∘Γ} arg f = var_{q∘γ} arg f

for any f ∈ C(Ȳ) which is holomorphic, locally independent of w, and nowhere zero on q ∘Γ. For by Proposition 3.2, f ∘q is constant on γ_{0}, and the contributions from γ_{1} and γ_{2} cancel each other, since q(t, 1) = q(t, e^{θ}). If f satisfies only the hypotheses of the lemma, we cannot of course guarantee that f is never zero on q ∘ Γ. But f(q ∘ γ_{0}) is a point, and f(q ∘ γ_{1}) ∪ f(q ∘ γ_{2}) is a countable union of smooth arcs, hence a set of measure zero. Therefore f(q ∘ Γ) is the union of f(q ∘ γ) and a compact set of measure zero, so there are infinitely many complex numbers a of arbitrarily small modulus such that f −a is nowhere zero on q ∘Γ. If we take any such a with modulus smaller than inf_{q∘γ} |f|, then we know that

S(f) = \frac{1}{2πi} var_{q∘Γ} arg(f − a).

Now if f ∘ q is a function of (z, |w|), then given any sufficiently small positive number ε (it suffices that ε < 2π/(θ + π) = 1/(m + 1/2)), we can write

f(q(z, w)) = f(q(z, |w|)) = g(z^{ε})

for some function g. When (z, w) runs through Γ, the point z^{ε} runs through a curve which we call Γ_{ε}, and which consists of the segment from 0 to 1 followed by the arc of the unit circle from 1 to e^{iεθ}, followed by the segment from that point back to the origin. The function g is continuous on the closure of the sector surrounded by Γ_{ε} and holomorphic inside the curve. And

var_{q∘Γ} arg(f − a) = var_{Γ_{ε}} arg(g − a) = 2πiN,

where N ⩾ 0 is the number of zeros of g−a inside the curve Γ_{ε}, by the usual argument principle. This proves the lemma.

Lemma 6.3. The set of all forms F = f dz ∧ dw/w ∈ O^{2}(Y) with f ∈ N(Y) is open in O^{2}(Y) and the functional S restricted to such forms is locally constant for the Bergman topology.

Proof. We know that point evaluations are continuous in the Bergman norm and moreover uniformly so over any compact set. Since the image of q ∘ γ is compact in Y, there is a constant C such that

sup_{q∘γ} |f| ⩽ C∥F ∥, F ∈ O^{2}(Y).

We apply this to G − F ∈ O^{2}(Y):

sup_{q∘γ} |g − f| ⩽ C∥G − F ∥.

For a fixed F with f ∈ N(Y) we define ε = inf_{q∘γ} |f|/C > 0. Then inf_{q∘γ} |g| > 0 for all G = g dz ∧ dw/w ∈ O^{2}(Y) such that ∥G − F ∥ < ε, i. e., we have proved that g ∈ N(Y). The change in argument for g must be the same as for f.

Theorem 6.4. Let Y be the smoothly bounded pseudoconvex subset of C×X_{θ} defined by (6.1), with θ = 2πm for some m = 2, 3, 4, ... . Then

F = z^{−1/m} dz ∧ dw/w

is an element of the Bergman space O^{2}(Y). Let φ ∈ C^{∞}(C) vanish near 0, satisfy 0 ⩽ φ ⩽ 1, and be such that

{z ∈ C; |z| < 2 & φ(z) ≠ 1}

has sufficently small Lebesgue measure. Then φF is a smooth form, and its Bergman projection is

P (φF) = h(z, w) dz ∧ dw/w

with a holomorphic function h which cannot be continuous up to the boundary of Y. We cannot use m = 1, for the function z^{−1} is not in L^{2} near the origin. Proof. We can calculate S(z^{−1/m}) explicitly, and find that it is equal to −1. Next we estimate the distance between P (φF) and F in the Bergman space: by the Pythagorean theorem,

∥P (φF) − F ∥^{2} = ∥φF − F ∥^{2} − ∥φF − P (φF)∥^{2} ⩽ ∥φF − F ∥^{2}.

Now ∥φF − F ∥ = ∥(1 − φ)F ∥ can be made arbitrarily small if only φ is suitably chosen. Lemma 6.3 shows that when ∥P (φF) − F ∥ is small enough, the coefficient h of P (φF) is in N(Y), and S(h) = S(z^{−1/m}) = −1.

We note that the coefficient h of P (φF) satisfies h(z, w) = h(z, |w|) in view of Proposition 3.1—or rather its analogue for O^{2}(Y). Now if h were continuous up to the boundary, we would have S(h) ∈ N by Lemma 6.2. This contradicts S(h) = −1 and proves the theorem.

References

Barrett, David. 1984. Irregularity of the Bergman projection on a smooth bounded

domain in C^{2}. Ann. of Math. 119, 431–436. Barrett, David. 1986. Biholomorphic domains with inequivalent boundaries. Inven-

tiones Math. 85, 373–377. Boas, Harold P. & Straube, Emil J. 1989. Complete Hartogs domains in C^{2} have

regular Bergman and Szegő projections. Math. Z. 201, 441–454. Bonami, Aline & Charpentier, Philippe. 1988. Une estimation Sobolev 1/2 pour le

projecteur de Bergman. C. R. Acad. Sci. Paris 307, Série I, 173–176. Diederich, Klas & Fornæss, John Erik. 1977. Pseudoconvex domains: An example

with nontrivial Nebenhülle. Math. Ann. 225, 275–292. Kiselman, Christer O. 1978. The partial Legendre transformation for plurisubhar-

monic functions. Inventiones Math. 49, 137–148. Krantz, Steven G. 1982. Function theory in several complex variables. John Wiley

& Sons.