Acta Mathematica Vietnamica 21:1, 69–94 Received May 4, 1995; in revised form July 1, 1995

Lineally convex Hartogs domains

Christer O. Kiselman

Contents:

1. Introduction

2. Weak lineal convexity

3. The non-local character of lineal convexity

4. Smooth vs. Lipschitz boundaries

5. Differential conditions

6. Differential conditions for Hartogs domains

7. Approximation of smoothly bounded lineally convex Hartogs domains 8. The non-local character of lineal convexity, revisited

9. Hartogs domains over a disk References

Resumo: Linie konveksaj Hartogs-aj regionoj

Ni studas linie konveksaj aroj de speciala tipo, nome Hartogs-aj regionoj, kaj pruvas ke tia regiono kun glata rando povas esti karakterizita per lokaj kondiĉoj.

Abstract: We study lineally convex domains of a special type, viz. Hartogs domains, and prove that such sets can be characterized by local conditions if they are smoothly bounded.

1. Introduction

Lineal convexity is a kind of complex convexity intermediate between usual convexity and pseudoconvexity. More precisely, if A is a convex set which is either open or closed, then A is lineally convex (this is true also in the real category), and if Ω is a lineally convex open set in C^{n}, the space of n complex variables, then Ω is pseudoconvex. Now pseudoconvexity is a local property in the sense that if any boundary point of an open set Ω has an open neighborhood ω such that Ω ∩ ω is pseudoconvex, then Ω is pseudoconvex; the analogous result holds for convexity. But it is well known that the property of lineal convexity is not a local property in this sense—for easy examples see section 3. The purpose of this paper is to investigate to what extent this is true for sets which are of a special form: the Hartogs domains.

Let us now give the main definition: a set A in C^{n} is said to be lineally convex if for every point b ∉ A there is a complex hyperplane passing through b but not intersecting A. In other words, the complement C^{n} ∖ A of A is a union of complex hyperplanes.

A lineally convex set whose boundary is sufficiently smooth satisfies a differential condition. Let ρ be a defining function for Ω, and let H and L denote, respectively, the Hessian and the Levi form at a boundary point a of Ω. Then the differential

This work was partially supported by the Swedish Natural Science Research Council. condition says that

(1.1)  |H(s)| ⩽ L(s) for all vectors s ∈ T_{C}(a),

where T_{C}(a) is the complex tangent space at the point a. See section 5 for details. Every lineally convex domain of class C^{2} satisfies the differential condition, but it is not known whether the converse is true. We shall prove that this is so in the special case of Hartogs domains, which we now proceed to define.

A Hartogs set in C^{n} × C is a set which contains, along with a point (z, t) ∈ C^{n} × C, also every point (z, t′) with |t′| = |t|. It is said to be a complete Hartogs set if it contains, with (z, t), also (z, t′) for all t′ with |t′| ⩽ |t|. Here we shall study open and bounded complete Hartogs sets; they are always defined by a strict inequality |t| < R(z), thus

(1.2)  Ω = {(z, t) ∈ C^{n} × C; |t| < R(z)},

where R is a real-valued function on C^{n}. Most of our results will be concerned with the case n = 1, thus

(1.3)  Ω = {(z, t) ∈ C × C; |t| < R(z)}.

Theorem 1.1. Let Ω be a bounded complete Hartogs domain in C^{2} with boundary of class C^{2}. If Ω satisfies the differential condition (1.1) at all boundary points, then Ω is lineally convex. Thus for complete Hartogs domains, the property of being lineally convex is a local property. Next we consider sets which are not smooth but of the special form

(1.4)  Ω = {(z, t) ∈ ω × C; |t| < R(z)}.

Here we assume R to be a C^{2} function, so that the differential condition makes sense for points (z, t) ∈ ∂ω with z ∈ ω, but the boundary is not smooth at the points (z, t) with z ∈ ∂ω. We shall say that Ω is a Hartogs domain over ω, or that ω is the base of Ω, if (1.4) holds with R > 0 in ω. In this case we prove:

Theorem 1.2. Let ω be a bounded open set in the complex plane C. If the closure of ω is not a disk, then lineal convexity over ω is not a local condition: we can find a Hartogs domain Ω over ω and two open sets ω_{0} and ω_{1} such that the Hartogs domains Ω_{j} over ω_{j} are lineally convex, j = 0, 1, but their union Ω = Ω_{0} ∪ Ω_{1} is not. If on the other hand ω is a disk, and Ω is a Hartogs domain of the form (1.4) satisfying the differential condition (1.1) at all boundary points over ω, then Ω is lineally convex.

Corollary 1.3. Let ω be an open set in C which is equal to the interior of its closure, and let Ω be a Hartogs domain over ω. Then the differential condition (1.1) imposed on all boundary points over ω is equivalent to lineal convexity if and only if ω is a disk.

I am grateful to Lê Hai Khôi for many helpful discussions concerning this paper.

2. Weak lineal convexity

There are several other notions related to lineal convexity. The property called weak lineal convexity is weaker than lineal convexity: an open connected set is called weakly lineally convex if through any boundary point there passes a complex hyperplane which does not intersect the set. An open set is said to be locally weakly lineally convex if through every boundary point a ∈ ∂Ω there is a complex hyperplane Y passing through a such that a does not belong to the closure of Y ∩ Ω. It is not difficult to prove that local weak lineal convexity implies pseudoconvexity.

For complete Hartogs sets it is very easy to see that weak lineal convexity implies lineal convexity:

Lemma 2.1. A complete Hartogs domain which is weakly lineally convex and has a lineally convex base is lineally convex.

Proof. Let (z^{0}, t^{0}) ∈ C^{n} ×C be an arbitrary point in the complement of Ω, a Hartogs domain defined by (1.2). If R(z^{0}) > 0, then the point (z^{0}, R(z^{0})t^{0}/|t^{0}|) belongs to ∂Ω, and if Ω is weakly lineally convex, there is a hyperplane passing through that point which does not cut Ω. Then the parallel plane through (z^{0}, t^{0}) does not cut Ω either. If R(z^{0}) ⩽ 0, then z^{0} does not belong to the base, and a hyperplane with equation ζ ·z = ζ ·z^{0} will do, since the base is lineally convex. This proves the lemma.

3. The non-local character of lineal convexity

The domain V = {(z, t) ∈ C^{2}; |t| < |z|} is easily seen to be lineally convex. Indeed, if (z_{0}, t_{0}) ∉ V with t_{0} ≠ 0, then the complex line {(z, t); z_{0}t = t_{0}z} passes through (z_{0}, t_{0}) and does not cut V; if on the other hand t_{0} = 0, we can for instance take the line {0} × C. A simple example of a domain which is locally lineally convex but not lineally convex can be built up from this set. Example 3.1. Define first

Ω_{+} = {(z, t); |z| < 1 and |t| < |z − 2|}; Ω_{−} = {(z, t); |z| < 1 and |t| < |z + 2|},

and then

Ω_{0} = Ω_{+} ∩ Ω_{−}; Ω_{1} = {(z, t) ∈ Ω_{0}; |t| < r},

where r is a constant with 2 < r < \sqrt{5}. All these sets are lineally convex. The two points (±i, \sqrt{5}) belong to the boundary of Ω_{0}; in the three-dimensional space of the variables (Re z, Im z, |t|), the set representing Ω_{0} has two peaks, which have been truncated in Ω_{1}. We now define Ω by glueing together Ω_{0} and Ω_{1}: define Ω as the subset of Ω_{0} such that (z, t) ∈ Ω_{1} if Im z > 0; we truncate only one of the peaks of Ω_{0}. The point (i − ε, r) for a small positive ε belongs to the boundary of Ω and the tangent plane at that point has the equation t = r and so must cut Ω at the point (−i + ε, r). Therefore Ω is not lineally convex, but it agrees with the lineally convex sets Ω_{0} and Ω_{1} when Im z < δ and Im z > −δ, respectively, for a small positive δ.

Proposition 3.2. Let ω_{0} and ω_{1} be two bounded open subsets in the complex plane such that none is contained in the closure of the other. Then there exists a Hartogs domain over ω = ω_{0} ∪ ω_{1} which is not lineally convex, but is such that the subsets Ω_{j} over ω_{j} are both lineally convex, j = 0, 1.

Proof. Take two points a ∈ ω_{1} ∖ \overline{ω}_{0} and b ∈ ω_{0} ∖ \overline{ω}_{1}, which exist by hypothesis. It is no restriction to assume that a = i, b = −i. Then take c > 0 so large that ω is contained in the disk of radius c − 1 and with center at the origin. We then define as in Example 3.1,

Ω_{0} = \{(z, t) ∈ C^{2}; |t| < |z ± c| and |t| < |z ± i(1 + \sqrt{c^{2} + 1} )|\},

and

Ω_{1} = {(z, t) ∈ Ω_{0}; |t| < r},

where r is a number slightly smaller than \sqrt{c^{2} + 1} but so close to that number that the peak that we have truncated in Ω_{1} near i lies outside ω_{0}, and the peak near −i lies outside ω_{1}. This is possible since we have assumed that i ∉ \overline{ω}_{0}, −i ∉ \overline{ω}_{1}, and Ω_{0} and Ω_{1} differ only above small neighborhoods of ±i which shrink to {±i} as r tends to \sqrt{c^{2} + 1}.

We now define Ω to agree with Ω_{j} over ω_{j}, j = 0, 1. The conclusion is as in Example 3.1.

4. Smooth vs. Lipschitz boundaries

The lineally convex set Ω_{0} constructed in Example 3.1 has the remarkable property that it cannot be approximated by lineally convex sets with smooth boundary. Its boundary, which is Lipschitz, cannot in any reasonable way be rounded off if we want to preserve lineal convexity. This is why we shall continue this investigation to see whether smoothly bounded sets admit a passage from the local to the global.

Before doing so, however, we shall illustrate the difference between domains which can be approximated by smoothly bounded lineally convex domains and those that have only Lipschitz boundary.

Let Ω be a complete Hartogs domain defined by (1.3) or (1.4) with R a function of class C^{1}. In the former case we define ω as the open set where R > 0. Often it will be convenient to use not R but h = R^{2} to define the set, thus, respectively,

(4.1)  Ω = {(z, t) ∈ C × C; |t| < R(z)} = {(z, t) ∈ C × C; |t|^{2} < h(z)},

and

(4.2)  Ω = {(z, t) ∈ ω × C; |t| < R(z)} = {(z, t) ∈ ω × C; |t|^{2} < h(z)}.

The complex tangent plane at a boundary point (z_{0}, t_{0}) with z_{0} ∈ ω has the equation

(4.3)  t − t_{0} = α(z − z_{0}), where α = \frac{h_{z}(z_{0})}{\overline{t_{0}}} = \frac{2t_{0}R_{z}(z_{0})}{R(z_{0})}.

Here and in the sequel we write h_{z} for the partial derivative ∂h/∂z, h_{z\overline{z}} for ∂^{2}h/∂z∂\overline{z}, etc. The tangent plane intersects the plane t = 0 in the point

(4.4)  b(z_{0}) = z_{0} − \frac{h(z_{0})}{h_{z}(z_{0})} = z_{0} − \frac{R(z_{0})}{2R_{z}(z_{0})}.

If R_{z}(z_{0}) = 0, the tangent plane has the equation t = t_{0}, and in this case we define b(z_{0}) = ∞, the infinite point on the Riemann sphere S^{2}.

Proposition 4.1. Let R ∈ C^{1}(C) and define Ω by (1.3). If Ω is bounded and lineally convex, then b(z), defined by (4.4), does not belong to ω, so that b is a continuous mapping from ω into S^{2} ∖ ω. Its range contains S^{2} ∖ \overline{ω}.

Proof. Clearly b is continuous as a mapping into C except where R_{z} = 0. Near such points, however, 1/b is continuous. The point (b(z_{0}), 0) cannot belong to Ω since Ω is lineally convex; thus b(z_{0}) ∉ ω. From every point (z, 0) outside the closure of Ω we can draw a tangent to Ω: this shows that the range of b contains C ∖ \overline{ω}; clearly it also contains ∞.

Corollary 4.2. If Ω is as in Proposition 4.1, then Ω is connected. The same is true if Ω is the union of an increasing family of bounded lineally convex sets Ω_{j} defined by functions R_{j} ∈ C^{1}(C).

Proof. Let ω_{1} be a connected component of ω. Then the image of ω_{1} under b contains S^{2} ∖ \overline{ω}_{1}. Since b(z_{0}) ∉ ω there can be no other component: we must have ω_{1} = ω. The statement about ⋃ Ω_{j} is now immediate.

Corollary 4.2 should be compared with the following easy result for Lipschitz boundaries.

Proposition 4.3. Given any open set ω in C there exists a Lipschitz continuous function R ∈ C(C) such that ω is the set where R is positive and the set Ω defined by R is lineally convex.

Proof. We define R(z) = inf_{a∉ω} |z − a|. The set Ω is lineally convex since it is an intersection of sets of the type V discussed in the beginning of section 3.

If a set does not have a boundary of class C^{1}, we cannot give a meaning to the notion of tangent plane. However, if the set is the union of an increasing family of sets with smooth boundaries, it is possible to use instead their tangent planes and then pass to the limit. Such limits of tangent planes can serve as well, as explained in the following easy lemma.

Lemma 4.4. Let Ω be the union of an increasing family of open lineally convex sets Ω_{j} with boundaries of class C^{1}. Let (j_{k}) be a sequence tending to +∞, and let Y_{k} be the complex tangent plane of ∂Ω_{j_{k}} at some point in the boundary of Ω_{j_{k}}, k ∈ N. Assume that Y_{k} converges to a hyperplane Y in the topology of hyperplanes. Then Y does not intersect Ω.

Proof. Suppose there is a point z ∈ Y ∩ Ω. Then also z ∈ Y ∩ Ω_{j_{k}} for all large k. Since Ω_{j_{k}} is open, there is a ball B(z, ε) ⊂ Ω_{j_{k}} for large k, say for k ⩾ k_{0}. But then Y_{k} intersects B(z, ε) for all large k, say for k ⩾ k_{1}. Thus Y_{k} ∩ Ω_{j_{k}} is non-empty for all k ⩾ max(k_{0}, k_{1}), contradicting the lineal convexity of Ω_{j_{k}}.

To recognize such limits of tangent planes we shall use the concept in the following definition.

Definition 4.5. Let X be any subset of C^{n} and a a point in the boundary ∂X. We shall say that a complex hyperplane Y is an admissible tangent plane to ∂X at a if there exists an open set A with boundary of class C^{1} such that A and X are disjoint, a belongs to the boundary of A, and Y is the complex tangent plane to A at a.

Proposition 4.6. Let Ω ⊂ C^{n} be the union of an increasing family of open sets Ω_{j} with boundaries of class C^{1}. Then any admissible tangent plane Y to ∂Ω is the limit of a sequence of tangent planes Y_{j} to ∂Ω_{j}. Therefore, in view of Lemma 4.4, Y cannot intersect Ω if the Ω_{j} are lineally convex.

Proof. Let a and A be as in Definition 4.5. By a coordinate change we may suppose that a = 0, that the real tangent plane to ∂A at the origin has the equation y_{n} = 0, and that A is defined by an inequality y_{n} > f(z_{1}, ..., z_{n−1}, x_{n}) near the origin for some function f of class C^{1}, which consequently vanishes at the origin together with its gradient. Write z′ = (z_{1}, ..., z_{n−1}) ∈ C^{n−1}. We then know that all points in Ω satisfy y_{n} < f(z′, x_{n}). Define g(z′, x_{n}) = f(z′, x_{n}) + |z′|^{2} + x_{n}^{2}, and let A_{c} be the set of all points such that y_{n} > g(z′, x_{n}) − c. We let c = c_{j} be the largest real number such that A_{c} and Ω_{j} are disjoint. Now 0 ∈ ∂Ω and Ω_{j} ↗ Ω; therefore we can be sure that c_{j} tends to zero as j → ∞. There is a point z^{j} which is common to the boundaries of A_{c_{j}} and Ω_{j}. Since A and Ω_{j} are disjoint, we have |(z^{j})′|^{2} + (x_{n}^{j})^{2} ⩽ c_{j}. The real tangent plane to ∂A_{c_{j}} at z^{j} is identical to the real tangent plane to ∂Ω_{j} at that point. We can control its slope, for the gradient of g is

grad g = grad f + grad(|z′|^{2} + x_{n}^{2}),

which is continuous and vanishes at the origin. Since ((z^{j})′, x_{n}^{j}) tends to the origin, this shows that the real tangent plane to ∂A_{c_{j}} at z^{j} must be close to the real hyperplane y_{n} = 0 if j is large, and then of course the complex tangent plane to ∂A_{c_{j}} at z^{j} is close to the complex hyperplane z_{n} = 0. The last statement now follows from Lemma 4.4.

If there are three points in a triangle on the boundary of a lineally convex set, certain values of the gradient at any of these points are forbidden as we can see from simple geometric considerations. In the space of three real variables (Re z, Im z, |t|) we can think of Ω as a banana and the tangent plane t = t_{0} + α(z − z_{0}) as a cone |t| = |t_{0} +α(z −z_{0})|; a cone of large opening cannot touch a banana everywhere. The next lemma expresses this in a precise way.

Lemma 4.7. Let R be the limit of an increasing sequence of functions R_{j} ∈ C^{1}(C) and assume that the sets Ω_{j} defined by R_{j} are lineally convex. Let three points 1, −1 and z_{0} = x_{0} + iy_{0} be such that R(1), R(−1), R(z_{0}) > 0 and assume that −1 < x_{0} < 1 and y_{0} > 0. Consider an admissible tangent plane of ∂Ω at the point (z_{0}, R(z_{0})) with the equation t = t_{0} + α(z − z_{0}) and assume that Im α is negative. Define

β = min(1 − |x_{0}|, y_{0}) > 0,
γ = R(z_{0})^{2} − min(R(1)^{2}, R(−1)^{2}) ∈ R.

Then α satisfies

(4.5)  (2 + y_{0}^{2})|α|^{2} − 2βR(z_{0})|α| + γ ⩾ 0.

This inequality will give us forbidden values of |α| provided Im α < 0, most easily if γ ⩽ 0, for then (4.5) implies that

(4.6)  |α| ⩾ \frac{2βR(z_{0})}{2 + y_{0}^{2}} as soon as Im α < 0.

But also when γ > 0 there are forbidden values. If we fix α such that 2βR(z_{0})|α| − (2 + y_{0}^{2})|α|^{2} > 0, then the lemma shows that

γ ⩾ 2βR(z_{0})|α| − (2 + y_{0}^{2})|α|^{2} > 0.

Thus it is impossible to obtain smaller values of γ. Proof of Lemma 4.7. By Proposition 4.6 the admissible tangent plane cannot cut the lineally convex set Ω, so in particular we must have

|t_{0} + α(±1 − z_{0})|^{2} ⩾ R(±1)^{2} ⩾ R(z_{0})^{2} − γ = t_{0}^{2} − γ.

Expanding the expression we find

|α|^{2}|± 1 − z_{0}|^{2} + 2t_{0} Re α(±1 − z_{0}) + γ ⩾ 0.

Now |± 1 − z_{0}|^{2} ⩽ 2 + y_{0}^{2} and Re α(±1 − z_{0}) = (±1 − x_{0}) Re α + y_{0} Im α, so that

|α|^{2}(2 + y_{0}^{2}) + γ ⩾ −2t_{0}((±1 − x_{0}) Re α + y_{0} Im α)

for both choices of sign. Noting that Im α is negative we obtain

|α|^{2}(2 + y_{0}^{2}) + γ ⩾ 2t_{0}((1 − |x_{0}|)| Re α| + y_{0}| Im α|) ⩾ 2βt_{0}|α|.

The lemma is proved.

Theorem 4.8. Let R be a function of class C^{1} or more generally a continuous function which is the limit of an increasing sequence of functions R_{j} of class C^{1} in the sets {z; R_{j}(z) > 0}. We assume that R is positive only in a bounded subset of the complex plane. The functions R_{j} define open sets Ω_{j}, which we assume to be lineally convex. Then the set M_{R} = {z; R(z) = sup_{w} R(w)} is convex.

Proof. Let a, b ∈ M_{R}. We have to prove that the whole segment [a, b] is contained in M_{R}. It is no restriction to assume that a = −1, b = 1. For every c ∈ [−1, 1] there must exist a point z ∈ ω with Re z = c, for otherwise ω would not be connected, in contradiction to Corollary 4.2. Thus sup_{Re z=c} R(z) > 0 for these c. Moreover this supremum must be equal to R(1) = sup R, for if there is a c ∈ [−1, 1] such that sup_{Re z=c} R(z) < R(1), then there must exist a saddle point z_{0} of R_{j} somewhere in the strip | Re z| < 1 with R_{j}(z_{0}) < R(1), and even R_{j}(z_{0}) < R_{j}(1) for j large. The gradient at a saddle point is zero, so that the tangent plane of ∂Ω_{j} at the boundary point (z_{0}, R_{j}(z_{0})) has the equation t = R_{j}(z_{0}) and cuts Ω_{j} at some point over 1 since R_{j}(z_{0}) < R_{j}(1). This contradicts the lineal convexity of Ω_{j}.

We thus have the situation that sup_{Re z=c} R(z) = R(1) for every c ∈ [−1, 1], which, since R is assumed continuous, means that there exists a point w with Re w = c and R(w) = R(1), thus w ∈ M_{R}. If Im w = 0 we are done: c = w ∈ M_{R}. If Im w ≠ 0, we may assume that Im w > 0; the other case is symmetric. We may also assume that | Im w| is minimal with this property, i.e., that the points z = c+iy with |y| < | Im w| do not belong to M_{R}.

Now Lemma 4.7 shows that the situation with these three points 1, −1 and w in M_{R} with Im w > 0 must lead to forbidden values of α at points near w. Most easily this is seen if R is of class C^{1}. We have R_{z}(w) = 0, so α = 2R_{z}(z_{0}) is small at all points near w; moreover, since R(x + iy) < R(w) for all y with 0 ⩽ y < w, there must exist points z_{0} = c + iy_{0} arbitrarily close to w with R_{y}(z_{0}) positive. Since R(z_{0}) ⩽ R(1), we have γ ⩽ 0 and (4.6) shows that all small values of |α| are forbidden.

When R itself is not of class C^{1} but a uniform limit of functions R_{j} of class C^{1} we must find an admissible tangent plane.

To produce points near w where there is an admissible tangent plane with Im α < 0 we define an auxiliary function

p_{a}(z) = R(1) + (x − c)^{2} + ε(y − \frac{1}{2} Im w)^{2} − a,

where ε > 0 and a is a real parameter. We have p_{a}(z) < R(1) only when

(x − c)^{2} + ε(y − \frac{1}{2} Im w)^{2} < a,

i.e., inside an ellipse, which we shall choose quite narrow. We fix ε > 0 and define b = ε(\frac{1}{2} Im w)^{2}. This implies that p_{b}(w) = p_{b}(c) = R(1), so that w and c are on the boundary of the domain p_{b} < R(1). If ε is small enough, then p_{b}(z) > R(z) when Im z ⩽ \frac{1}{2} Im w. Since p_{b}(w) = R(w), the inequality p_{a} ⩾ R implies a ⩽ b. We now choose a as the largest real number such that p_{a} ⩾ R; we must then have 0 < a ⩽ b. Moreover there must exist a point z_{0} such that p_{a}(z_{0}) = R(z_{0}) in view of the maximality of a, and we know that Im z_{0} > \frac{1}{2} Im w, ensuring that the imaginary part of α = 2p_{a,z} = p_{a,x} − ip_{a,y} is negative: Im α = −2ε(y_{0} − \frac{1}{2} Im w) < 0. We also note that

| Im α| = 2ε|y_{0} − \frac{1}{2} Im w| ⩽ ε Im w and | Re α| = 2|x_{0} − c| ⩽ 2\sqrt{a} ⩽ \sqrt{ε} Im w

are arbitrarily small. Thus |α| is as small as we like, which contradicts (4.6). The next result describes a situation in contrast to Theorem 4.8:

Proposition 4.9. Given any closed set M in the complex plane such that its complement is a union of open disks of radius ε there exists a Lipschitz continuous function R such that M_{R} = M and the domain Ω defined by (4.1) with this R is lineally convex.

Proof. Define R(z) = min(ε, inf_{a∈A} |z − a|), where A is the set of all centers of disks of radius ε in the complement of M.

5. Differential conditions

Let Ω be an open set in C^{n} with boundary of class C^{1}. Then there exists a function ρ ∈ C^{1}(C^{n}), called a defining function, such that dρ ≠ 0 wherever ρ = 0 and

Ω = {z ∈ C^{n}; ρ(z) < 0}.

The complex tangent space at a point a on the boundary of Ω is defined by

∑_{j=1}^{n} \frac{∂ρ}{∂z_{j}} (a)s_{j} = 0.

We shall denote it by T_{C}(a). The real tangent space is defined by

Re ∑_{j=1}^{n} \frac{∂ρ}{∂z_{j}} (a)s_{j} = 0

and will be denoted by T_{R}(a). If ρ is of class C^{2} we define the Hessian form of ρ as the quadratic form

H(s) = ∑_{j,k=1}^{n} \frac{∂^{2}ρ}{∂z_{j}∂z_{k}} (a)s_{j}s_{k}, s ∈ C^{n},

and the Levi form of ρ as the sesquilinear form

L(s) = ∑_{j,k=1}^{n} \frac{∂^{2}ρ}{∂z_{j}∂\overline{z}_{k}} (a)s_{j}\overline{s}_{k}, s ∈ C^{n}.

Definition 5.1. We shall say that a set Ω with boundary of class C^{2} satisfies the differential condition at a boundary point a of Ω if

(5.1)  |H(s)| ⩽ L(s) for all vectors s ∈ T_{C}(a).

We shall say that Ω satisfies the strong differential condition at a if we have

(5.2)  |H(s)| < L(s) for all s ∈ T_{C}(a) ∖ {0}.

These conditions should be compared with the differential condition for convexity: |H(s)| ⩽ L(s) for all vectors s in the real tangent space T_{R}(a). This is a local condition, and it is well known that it is equivalent to convexity of Ω. The proof of this fact most conveniently goes via approximation of the set by sets satisfying the corresponding strong condition, i.e., |H(s)| < L(s) for all s ∈ T_{R}(a) ∖ {0}.

The following two lemmas are well known (cf. Zinov′ev [1971] and Hörmander [1994, Corollary 4.6.5]). We include them for ease of reference.

Lemma 5.2. Let Ω be an open subset of C^{n} with boundary of class C^{2}. If Ω is locally weakly lineally convex, then Ω satisfies the differential condition at every boundary point.

Proof. Let a be an arbitrary boundary point of a locally weakly lineally convex open set Ω. Then there exists a complex hyperplane through a which does not cut Ω close to a. This hyperplane cannot be anything but T_{C}(a) since the boundary is of class C^{1}. Therefore if we take an arbitrary vector s ∈ T_{C}(a) and consider the function φ(t) = ρ(a + ts) of a real variable t, its second derivative must be non-negative at the origin. If we express the condition φ′′(0) ⩾ 0 in terms of H and L we get Re H(s) + L(s) ⩾ 0, which, since H is quadratic and L sesquilinear, is equivalent to |H| ⩽ L.

Lemma 5.3. Let Ω be an open subset of C^{n} with boundary of class C^{2}. If Ω satisfies the strong differential condition at every boundary point, then Ω is locally weakly lineally convex.

Proof. With φ as in the proof of the previous lemma we must have φ′′(0) > 0 if Ω satisfies the strong differential condition. This imples that T_{C}(a) cannot cut Ω close to a.

It is known that if Ω is a connected open set with boundary of class C^{1} which is locally weakly lineally convex, then Ω is weakly lineally convex; see, e.g., Hörmander [1994, Proposition 4.6.4]. We shall come back to this result in section 7.

6. Differential conditions for Hartogs domains

In this section we shall see what the differential conditions look like in the case of a complete Hartogs domain in C^{2}. Let Ω be a complete Hartogs domain in C^{2} defined by (4.1). If h is of class C^{1}, we can choose as its defining function

ρ(z, t) = t\overline{t} − h(z).

It must satisfy d′ρ ≠ 0 when ρ = 0, which means that d′ρ = \overline{t}dt − h_{z}dz ≠ 0 when |t|^{2} = h(z). Since the first term of d′ρ is \overline{t}dt, which is non-zero everywhere except in the plane t = 0, the only condition is that h_{z} ≠ 0 when h = 0, i.e., that h itself shall be a defining function in C. It defines a subset ω of the complex plane over which Ω is situated.

Lemma 6.1. Let h be a defining function of an open set ω in C of class C^{k}, k ⩾ 1. Then the complete Hartogs domain in C^{2} defined by (4.1) has boundary of class C^{k}. When k ⩾ 2, it satisfies the differential condition at every boundary point if and only if h satisfies the condition

(6.1)  \frac{|h_{z}|^{2}}{h} ⩾ h_{z\overline{z}} + |h_{zz}| wherever h > 0.

Furthermore Ω satisfies the strong differential condition if and only if there is strict inequality in (6.1).

Proof. Let us look at the Hessian and Levi forms of ρ(z, t) = |t|^{2} − h(z). They are, respectively,

H(s) = −h_{zz}s_{1}^{2} and L(s) = −h_{z\overline{z}}|s_{1}|^{2} + |s_{2}|^{2}, s = (s_{1}, s_{2}) ∈ C^{2}.

The differential condition |H| ⩽ L takes the form

|h_{zz}||s_{1}|^{2} ⩽ −h_{z\overline{z}}|s_{1}|^{2} + |s_{2}|^{2} for all s ∈ T_{C}(a).

The tangent plane is defined by −h_{z}s_{1} + \overline{t}s_{2} = 0. When t ≠ 0 we use this equation to eliminate s_{2}: the condition takes the form (6.1). Near t = 0 we eliminate instead s_{1} and get

(h_{z\overline{z}} + |h_{zz}|)\frac{h}{|h_{z}|^{2}} ⩽ 1.

This inequality is satisfied, even strictly, at all boundary points sufficiently close to t = 0, provided h_{z} ≠ 0 near h = 0. Therefore, if h is a defining function for ω, then ρ is a defining function for Ω and condition (6.1) implies the differential condition at all boundary points of Ω, including those where t = 0. Conversely, if ρ is a defining function for Ω, then h is a defining function for ω, and the differential condition for Ω implies the condition (6.1) for h. Remark 6.2. We can of course express the differential condition (6.1) in terms of the radius R = \sqrt{h}. It becomes

(6.2)  |R_{z}|^{2} ⩾ |R_{z}^{2} + RR_{zz}| + RR_{z\overline{z}},

which is less convenient to work with than (6.1). If h is concave, then h_{z\overline{z}} +|h_{zz}| ⩽ 0, so that (6.1) holds. More generally, if R is concave, then R_{z\overline{z}} + |R_{zz}| ⩽ 0, which implies that (6.2) holds. It is also possible to express the differential condition in terms of the function f = − log R. It then takes the form

(6.3)  |f_{zz} − 2f_{z}^{2}| ⩽ f_{z\overline{z}}.

In Kiselman [ms] I have studied convexity properties of this function f.

7. Approximation of smoothly bounded lineally convex Hartogs domains

Theorem 7.1. Let

(7.1)  Ω = {(z, t) ∈ C^{2}; |t| < R(z)}

be a bounded complete Hartogs domain in C^{2} with boundary of class C^{2}. Suppose Ω satisfies the differential condition at all boundary points. Then Ω can be approximated from the inside by Hartogs domains

Ω_{ε} = {(z, t); |t| < R_{ε}(z)}

which satisfy the strong differential condition at all boundary points (z, t) except those where R_{z}(z) = 0. In fact, we can take R_{ε} = \sqrt{R^{2} − ε} with ε positive and small enough.

Proof. Of course we should not try to do any calculations with R, but use R^{2} = h instead. The differential condition (6.1) contains the value of h only at one place, and h_{ε} = h − ε has the same derivatives as h, so we can write

\frac{|h_{z}|^{2}}{h − ε} > \frac{|h_{z}|^{2}}{h} ⩾ h_{z\overline{z}} + |h_{zz}|

except of course when h_{z} = 0. Thus the boundary of Ω_{ε} satisfies the strong differential condition except at the points where h_{z} = 0. So far the argument is valid for all positive ε. We need to check that h_{ε} is a defining function; otherwise we cannot apply Lemma 6.1. But the gradient of h_{ε} is the same as that of h, which is non-zero when h = 0, hence also when h_{ε} = 0, provided ε is small enough. Thus h_{ε} is a defining function for all small ε, proving the theorem.

We shall now see that the approximating sets Ω_{ε} that we constructed in Theorem 7.1 are in fact lineally convex. Let us agree to say that a complex plane with the equation z = constant is vertical and a plane with the equation t = constant is horizontal.

Proposition 7.2. Let Ω be a bounded complete Hartogs domain in C^{2} with boundary of class C^{2} satisfying the strong differential condition except possibly at the points where the tangent plane is horizontal. Then Ω is lineally convex. We shall need the following three lemmas.

Lemma 7.3. Let Ω be as in Proposition 7.2 and let L be a complex line in C^{2} which is not horizontal. Then L ∩ Ω consists of a finite number of open sets bounded by C^{2} curves obtained as transversal intersections of L and ∂Ω (and L ∩ ∂Ω consists of these curves plus a finite number of isolated points).

Proof. Take an arbitrary boundary point a and let L be a complex line through a which is not horizontal. If L is the tangent plane, L = a + T_{C}(a), then the proof of Lemma 5.3 shows that L intersects \overline{Ω} near a only in the point a. If, on the other hand, L is not the tangent plane, then L ∩ (a + T_{C}(a)) ≠ L, so ∂Ω cuts L transversally, and ∂Ω∩L is a C^{2} curve in L near a. Thus L∩∂Ω consists of a number of C^{2} curves plus isolated points—by compactness there can only be finitely many curves and points.

Lemma 7.4. Let Ω and L satisfy the hypotheses of the previous lemma. Then Ω ∩ L is connected, and Ω ∩ (a + T_{C}(a)) is empty for all a ∈ ∂Ω.

Proof. We shall follow closely the proof of Proposition 4.6.4 in Hörmander [1994]— we only have to be careful to avoid horizontal planes. Let (z_{j}, t_{j}), j = 0, 1, be two points in L ∩ Ω. We have to prove that they belong to the same component of L ∩ Ω. Suppose first that both t_{0} and t_{1} are non-zero. Since Ω is connected, there is a curve γ which goes from γ(0) = (z_{0}, t_{0}) to γ(1) = (z_{1}, t_{1}). We can actually do this in such a way that the complex line L_{s} which contains γ(0) and γ(s), 0 < s ⩽ 1, is never horizontal. Indeed, we first go from (z_{0}, t_{0}) to (z_{0}, 0) along a curve in the plane z = z_{0} avoiding (z_{0}, t_{1}); then along a curve in the plane t = 0 from (z_{0}, 0) to (z_{1}, 0); and then finally from (z_{1}, 0) to (z_{1}, t_{1}) along a curve in the plane z = z_{1} avoiding (z_{1}, t_{0}). (We know that t_{0} ≠ t_{1}.) Thus none of the lines L_{s} is horizontal, and we can apply Lemma 7.3 to them. Consider the set C of all s ∈ ]0, 1] such that γ(0) and γ(s) belong to the same component of L_{s} ∩ Ω. Then certainly C contains all sufficiently small numbers, for γ(0) and γ(s) are then in the line z = z_{0}, whose intersection with Ω is a disk. The set C is open as a subset of ]0, 1] in view of Lemma 7.3, but so is its complement with respect to ]0, 1]. Since it is non-empty, it must contain 1, i.e., (z_{0}, t_{0}) and (z_{1}, t_{1}) belong to the same component of L ∩ Ω. If one of t_{0}, t_{1} is zero, we choose a point with non-zero second coordinate in the neighborhood and argue as above.

Consider now a tangent plane L = a+T_{C}(a) and planes L_{ε} = a_{ε} +T_{C}(a) parallel to it, where we write a_{ε} = (z_{0}, (1−ε)t_{0}) if a = (z_{0}, t_{0}). We already know from Lemma 5.3 that L cannot intersect Ω close to a. However, it cannot cut Ω at all, for if it did, then a parallel plane L_{ε} for some small positive ε would intersect Ω in a component close to a and another nonempty set at some distance from a, thus in a disconnected set. This proves Lemma 7.4.

Lemma 7.5. Let Ω be as in Proposition 7.2 and let a ∈ ∂Ω be such that the tangent plane is horizontal. Then Ω ∩ (a + T_{C}(a)) is empty; in other words R has a global maximum at a. Consequently any horizontal plane L intersects Ω in finitely many open sets bounded by C^{2} curves obtained as transversal intersections of L by ∂Ω.

Proof. Let (z_{0}, t_{0}) be a boundary point such that the tangent plane is horizontal, i.e., R_{z}(z_{0}) = 0. Suppose the tangent plane cuts Ω in some point (z_{1}, t_{1}). We must then have t_{1} = t_{0}. Since Ω and its base ω are connected, we can find a curve γ in ω connecting z_{0} to z_{1}, say γ(s) = z_{s}, s ∈ [0, 1]. Consider now the tangent planes at the points (z_{s}, R(z_{s})); we denote them by L_{s} = (z_{s}, R(z_{s})) + T_{C}(z_{s}, R(z_{s})). It is no restriction to assume t_{0} > 0, so that R(z_{0}) = t_{0}. We know that L_{0} is horizontal, but certainly not all the L_{s} can be horizontal, since R(z_{1}) > |t_{1}| = |t_{0}| = R(z_{0}). Let s_{0} be the infimum of all s such that L_{s} is not horizontal; we must have 0 ⩽ s_{0} < 1. The planes L_{s} with 0 ⩽ s ⩽ s_{0} are identical and all intersect Ω in the point (z_{1}, t_{1}). It is now clear that there exists a tangent plane L_{s} with s just a little bit larger than s_{0} which is not horizontal and still cuts Ω. This contradicts Lemma 7.4. Proof of Proposition 7.2. We know from Lemma 7.4 that a tangent plane which is not horizontal does not intersect Ω; we obtain the same conclusion from Lemma 7.5 for a horizontal tangent plane. Thus Ω is weakly lineally convex. Lemma 2.1 shows that this implies lineal convexity.

We can now finally state:

Theorem 7.6. Let Ω be a bounded complete Hartogs domain in C^{2} with boundary of class C^{2}. If Ω satisfies the differential condition (5.1) at all boundary points, then Ω is lineally convex.

Proof. Using Theorem 7.1 we construct open sets Ω_{ε} which tend to Ω. Also, if R(z_{0}) > 0, the tangent plane of ∂Ω_{ε} at (z_{0}, \sqrt{R(z_{0})^{2} − ε}) tends to that of ∂Ω at (z_{0}, R(z_{0})). The sets Ω_{ε} are lineally convex by Proposition 7.2. Then also their limit Ω is lineally convex. Indeed, if a tangent plane to ∂Ω intersected Ω, then it would cut also Ω_{ε} for all sufficiently small ε, and then also for ε small enough the corresponding tangent plane to ∂Ω_{ε} would cut Ω_{ε}. This is a contradiction.

8. The non-local character of lineal convexity, revisited

Having settled the question of lineal convexity of smoothly bounded Hartogs domains we now turn to sets of the form

(8.1)  Ω = {(z, t) ∈ ω × C; |t| < R(z)} = {(z, t) ∈ ω × C; |t|^{2} < h(z)},

where ω is a given open set in C and h is a C^{2} function in the closure of ω satisfying h > 0 and the differential condition (6.1). Its boundary is smooth enough over points in ω, but is only Lipschitz at points over ∂ω. It turns out that when ω is a disk, then the differential condition implies lineal convexity: we shall study this question in section 9. On the other hand, if ω is a set such that \overline{ω} is not a disk, then the differential condition does not imply lineal convexity. This is the topic of the present section.

The property of being a disk is invariant under Möbius mappings, and disks are the only sets which remain convex under all Möbius mappings. This is a kind of explanation for the phenomenon we encounter here, and it is therefore natural to study how the differential condition (6.1) behaves under Möbius mappings. This is explained in the next lemma.

Lemma 8.1. Let Ω be a Hartogs domain in C^{2} defined by |t| < R(z), let a, b, c, d be four complex numbers with ad − bc ≠ 0, and let Ω_{1} be the Hartogs domain defined by |t| < R_{1}(z) = |c + dz|R((a + bz)/(c + dz)). Then Ω and Ω_{1} are lineally convex simultaneously. The two functions h and h_{1}(z) = |c + dz|^{2}h((a + bz)/(c + dz)) satisfy the differential condition (6.1) simultaneously.

Proof. Consider the mapping

(C ∖ {0}) × C × C ∋ (z_{0}, z_{1}, t) ↦ (z_{1}/z_{0}, t/z_{0}) ∈ C^{2}.

Under it the pull-back of the hyperplane c + ζz + τt = 0 is the hyperplane cz_{0} + ζz_{1} + τt = 0. It follows that the pull-back of a lineally convex set in C^{2} is a complex homogeneous lineally convex set in C^{3}. Now any linear mapping of the form

C^{3} ∋ (z_{0}, z_{1}, t) ↦ (cz_{0} + dz_{1}, az_{0} + bz_{1}, t) ∈ C^{3}

with ad − bc ≠ 0 preserves lineal convexity, and mappings

C^{3} ∋ (z_{0}, z_{1}, t) ↦ (1, \frac{az_{0} + bz_{1}}{cz_{0} + dz_{1}} , \frac{t}{cz_{0} + dz_{1}} ) ∈ C^{3}

preserve lineally convex sets which are complex homogenoeus. If we transport this back to C^{2} we get a mapping of the form

(z, t) ↦ (\frac{a + bz}{c + dz} , \frac{t}{c + dz} ) .

This proves that Ω and Ω_{1} as defined in the statement of the lemma are lineally convex at the same time. The statement about the differential condition for h and h_{1} can be verified directly, perhaps easiest if we check it for the special mappings z ↦ c + dz and z ↦ 1/z, which together generate all Möbius mappings.

Lemma 8.2. Let K be a compact subset of C with connected complement. Assume that K is not a disk. Then there exists a closed disk D_{1} containing K such that K ∩ ∂D_{1} has at least two components.

Proof. Let D_{0} be the closed disk of minimal radius which contains K. By hypothesis K ≠ D_{0} and C ∖ K is connected, so there exists a point a_{0} ∈ ∂D_{0} ∖ K. Let H be an open halfplane which contains K but is such that a_{0} ∉ \overline{H}. Now consider the closed disk D_{1} of minimal radius among those that contain K and have ∂H as a tangent. We claim that there are four points a, b, c, d ∈ ∂D_{1} which are in that order along the circumference and with a, c ∉ K, b, d ∈ K. This will show that b and d belong to different components of K ∩ ∂D_{1}. To find these points we argue as follows. Let a be the point of ∂D_{1} at which ∂H is tangent; thus a ∈ ∂D_{1} and a ∉ K. Next, D_{1} ⊄ D_{0}, so there is a point c ∈ ∂D_{1} ∖ D_{0}. Thus c ∉ K. Finally we claim that there are two points b, d ∈ ∂D_{1} ∩ K on either side of the segment [a, c]. This is so because if one of the arcs from a to c were disjoint from K, then it can easily be seen that D_{1} would not be minimal among the disks that contain K and are tangent to ∂H. This completes the proof.

Theorem 8.3. Let ω be a bounded connected open subset of C such that the complement S^{2} ∖ \overline{ω} of its closure with respect to the Riemann sphere S^{2} = C ∪ \{∞\} has at least one component which is not a disk. Then there exists a Hartogs domain defined by a smooth function and with base ω such that it is not lineally convex, although ω = ω_{0} ∪ ω_{1} and the Hartogs domain over ω_{j} is lineally convex, j = 0, 1. In particular the function defining Ω satisfies the differential condition (6.1).

Proof. Let K be the complement of a component of S^{2} ∖ \overline{ω} which is not a disk; thus K contains \overline{ω}. Moreover the complement of K is connected and ∂K ⊂ ∂ω. We may assume that K is compact: if not we use a Möbius mapping to reduce ourselves to that case. Let a, b, c, d ∈ ∂D_{1} be the four points whose existence is guaranteed by Lemma 8.2; recall that b, d ∈ K and a, c ∉ K. Now take a new closed disk D_{2} which does not contain a, b, or d, but contains c in its interior, and is so close to D_{1} that b and d belong to different components of K ∖ D_{2}. This is possible because a does not belong to K. Now we map D_{2} onto the closed right halfplane, taking a to 0 and some point outside K and near c to infinity. We are thus reduced to a situation where K is still compact in C, whereas ∂D_{2} is the imaginary axis, with a = 0 and Im b and Im d of different signs, say for definiteness Im b < 0 and Im d > 0. Moreover we can take D_{2} so close to D_{1} that the points in K which are not in D_{2} are never real. Then we can define a function R as follows. First take a smooth concave function ψ of a real variable such that ψ(s) = 1 when s ⩾ 0 and ψ(s) < 1 for s < 0, but still so that ψ(Re z) > 0 for all points z ∈ \overline{ω}. Then define

R(z) = \{ \begin{matrix}ψ(Re z) & when z ∈ ω, Re z < 0, Im z < 0, \\ 1 & at other points in ω. \end{matrix}

This function is continuous, even identically one, in a neighborhood of the intersection of ω and the real axis.

The tangent plane at a point (z_{0}, t_{0}) ∈ ∂Ω with z_{0} ∈ ω has the equation (4.3). In particular, we may take t_{0} = R(z_{0}) and get

t = R(z_{0}) + 2R_{z}(z_{0})(z − z_{0}).

In the present case R is locally a function of Re z, say R(z) = k(x), so that R_{z} = k_{x}/2 is real. Thus the tangent plane is

t = R(z_{0}) + k_{x}(x_{0})(z − z_{0}) = R(z_{0}) + k_{x}(x_{0})(x − x_{0}) + ik_{x}(x_{0})(y − y_{0}),

and, writing z = z_{0} + z_{1}, we obtain

|t|^{2} = R(z_{0})^{2} + 2k_{x}(x_{0})R(z_{0})x_{1} + k_{x}(x_{0})^{2}x_{1}^{2} + k_{x}(x_{0})^{2}y_{1}^{2}.

When x_{1} < 0 and k_{x}(x_{0}) is positive and small,

|t|^{2} ≈ R(z_{0})^{2} + 2k_{x}(x_{0})R(z_{0})x_{1} < R(z_{0})^{2}.

Since ω is connected and has the point b on its boundary, we can choose z_{0} such that y_{0} < 0 and x_{0} < 0 with k_{x}(x_{0}) arbitrarily small, so small that indeed |t| < R(z_{0}). Then we choose z = z_{0} + z_{1} ∈ ω with Im z > 0. Thus R(z) = 1, so the tangent plane at (z_{0}, R(z_{0})) cuts Ω in a point above z. This proves that Ω is not lineally convex. However, if we look at the parts of ω where Im z > −ε and Im z < ε respectively, then R is the restriction of a globally concave function in each of them and therefore defines a lineally convex set.

Theorem 8.4. Let ω be a bounded open set in C such that S^{2} ∖ \overline{ω} is not connected. Then there is a function h ∈ C^{∞}(\overline{ω}), h > 0, which satisfies the differential condition (6.1) but is such that the Hartogs domain it defines over ω is not lineally convex.

Proof. If one of the components of S^{2} ∖ \overline{ω} is not a disk, we already know the result by Theorem 8.3. The case when all components of S^{2} ∖ \overline{ω} are disks remains to be considered. This means that \overline{ω} is a disk from which countably many disks (at least one) have been removed. Any one of these holes can be moved by a Möbius transformation so that it becomes concentric with the outer circumference of \overline{ω}; in other words \overline{ω} is an annulus r_{0} ⩽ |z| ⩽ r_{1} from which possibly a number of disks have been removed. It is clearly enough to consider the case of the annulus, for the possible presence of other holes will not destroy our conclusion.

So assume \overline{ω} is the annulus r_{0} ⩽ |z| ⩽ r_{1} and define R_{0}(z) = 1−ax^{2} −by^{2}, where 0 < a < b and b is so small that R_{0} > 0 in \overline{ω}. Next define φ to be a concave C^{∞} function of one real variable such that φ(s) = s for all s ⩽ 1 − br_{0}^{2} + ε and φ(s) = c when s ⩾ 1 − ar_{0}^{2} − ε for some positive ε and a suitable constant c; by necessity we must have c < 1 − ar_{0}^{2}. Define R_{1}(z) = φ(R_{0}(z)). We observe that R_{0} = R_{1} in a neighborhood of the intersection of the imaginary axis and \overline{ω}. Both R_{0} and R_{1} are concave in C, so the corresponding Hartogs domains over |z| < r_{1} are convex and therefore lineally convex. It follows that the Hartogs domains over ω are lineally convex. Now define R to agree with R_{0} in the right halfplane and with R_{1} in the left halfplane. Note that R(z) = R_{1}(z) = c at points z ∈ ω close to −r_{0}, so that the tangent plane at a boundary point over such a point has the equation t = t_{0} with |t_{0}| = c < 1 − ar_{0}^{2}. But over a point z in ω close to r_{0} we have R(z) = R_{0}(z) > c, so the tangent plane t = t_{0} cuts Ω. This proves that Ω cannot be lineally convex.

9. Hartogs domains over a disk

The differential condition over a disk remains to be studied. We shall see that it is then equivalent to lineal convexity.

We shall write D(c, r) for the open disk in the complex plane with center c and radius r, and just D for the open unit disk D(0, 1).

Proposition 9.1. Let h ∈ C^{2}(D), h > 0, be a real-valued function which satisfies the differential condition

(9.1)  \frac{|h_{z}|^{2}}{h} ⩾ h_{z\overline{z}} + |h_{zz}|,  |z| < 1.

Let φ ∈ C^{2}(R) be real-valued, decreasing and satisfy φ ⩽ 1 everywhere and φ′′ < 0 wherever φ < 1. Assume that there are constants a and A such that

(9.2)  Re [\frac{2zh_{z}(z)}{h(z)}] ⩽ a < 1

and

(9.3)  |\frac{2zh_{z}(z)}{h(z)}| ⩽ A < +∞

wherever 0 < φ(z\overline{z}) < 1. Then g(z) = φ(z\overline{z})h(z) satisfies the differential condition wherever φ(z\overline{z}) > 0 and |z| < 1, provided φ′/φ′′ is small enough, more precisely if either A ⩽ 1 or else

\frac{φ′(s)}{sφ′′(s)} ⩽ \frac{2(1 − a)}{A^{2} − 1} when s is such that 0 < φ(s) < 1.

Proof. With g(z) = φ(z\overline{z})h(z) we have

g_{z} = φ′\overline{z}h + φh_{z},
g_{zz} = φ′′\overline{z}^{2}h + 2φ′\overline{z}h_{z} + φh_{zz},
g_{z\overline{z}} = φ′′|z|^{2}h + φ′h + 2φ′ Re zh_{z} + φh_{z\overline{z}}.

Thus what we have to prove is, writing r for |z|,

\frac{|φ′\overline{z}h + φh_{z}|^{2}}{φh} ⩾ r^{2}φ′′h + φ′h + 2φ′ Re zh_{z} + φh_{z\overline{z}} + |φ′′\overline{z}^{2}h + 2φ′\overline{z}h_{z} + φh_{zz}|.

We expand the left-hand side and find that the term 2φ′ Re zh_{z} appears on both sides. We shall therefore prove

\frac{r^{2}φ^{′2}h}{φ} + \frac{φ|h_{z}|^{2}}{h} ⩾ r^{2}φ′′h + φ′h + φh_{z\overline{z}} + |φ′′\overline{z}^{2}h + 2φ′\overline{z}h_{z} + φh_{zz}|.

This formula follows from |h_{z}|^{2}/h ⩾ h_{z\overline{z}} + |h_{zz}|, which holds by hypothesis, and

(9.4)  \frac{r^{2}φ^{′2}h}{φ} ⩾ r^{2}φ′′h + φ′h + |φ′′\overline{z}^{2}h + 2φ′\overline{z}h_{z}|,

which we shall prove now. We divide both sides of this inequality by the positive quantity −r^{2}φ′′h (if φ′′ is zero there is nothing to prove), and find the equivalent inequality

−\frac{φ^{′2}}{φφ′′} ⩾ −1 − \frac{φ′}{r^{2}φ′′} + |−\frac{\overline{z}^{2}}{r^{2}} − 2\frac{φ′\overline{z}h_{z}}{r^{2}φ′′h}| = −1 − \frac{φ′}{r^{2}φ′′} + |1 + \frac{φ′}{r^{2}φ′′} \frac{2zh_{z}}{h} | .

Since −φ^{′2}/φφ′′ is positive, it suffices to prove that

1 + t ⩾ |1 + tw| when t = \frac{φ′(r^{2})}{r^{2}φ′′(r^{2})} and w = \frac{2zh_{z}(z)}{h(z)}.

This inequality in turn follows from

(1 + t)^{2} ⩾ |1 + tw|^{2} = 1 + 2t Re w + t^{2}|w|^{2},

which holds as soon as 2 + t ⩾ 2 Re w + t|w|^{2}. By hypothesis Re w ⩽ a < 1 and |w| ⩽ A, so (9.4) follows as soon as either A ⩽ 1 or else A > 1 and t ⩽ 2(1 − a)/(A^{2} − 1). This proves the proposition.

Example 9.2. As an example of the function φ in Proposition 9.1 we let s_{0} be an arbitrary number such that 0 < s_{0} < 1 and take a smooth function φ satisfying φ(s) = 1 for s ⩽ s_{0} and whose derivative is φ′(s) = −C exp(−1/(s − s_{0})) for s > s_{0}. Then we determine C to make φ(1) = 0; this means that we choose C to satisfy

C ∫_{s_{0}}^{1} e^{−1/(s−s_{0})}ds = 1.

We note that φ′(s)/sφ′′(s) = (s − s_{0})^{2}/s, which varies between 0 and (1 − s_{0})^{2}. Thus if 1 − s_{0} is small enough, we can conclude that the new function φ(z\overline{z})h(z) satisfies the differential condition (9.1) over the open unit disk and it agrees with h when |z| ⩽ \sqrt{s_{0}}.

We need to study condition (9.2) more closely. In fact it has a simple geometric meaning.

Definition 9.3. Let a complete Hartogs domain

Ω = {(z, t) ∈ ω × C; |t|^{2} < h(z)}

be defined over a bounded domain ω in C by a function h ∈ C^{1}(ω), h > 0. Denote by (b(z), 0) the point at which the tangent at a point (z, t) ∈ ∂Ω with z ∈ ω intersects the plane t = 0 (put b(z) = ∞ if there is no such point). We shall say that Ω satisfies the tangent condition if

inf_{z∈ω} d(b(z), ω) > 0,

where d denotes the distance from a point to a set. If Ω is defined by a function R ⩾ c > 0 and is lineally convex, then it must satisfy the tangent condition, but not only that—we can deduce important quantitative information from its lineal convexity:

Lemma 9.4. Let R ∈ C^{1}(ω) be such that the set Ω defined by (8.1) is lineally convex. Then

(9.5)  inf_{z∈ω} d(b(z), ω) ⩾ \frac{inf_{ω} R}{2 sup_{ω} |R_{z}|} ⩾ \frac{inf_{ω} h}{sup_{ω} |h_{z}|}.

If R ⩾ c > 0 in ω, then Ω satisfies the tangent condition.

Proof. The tangent plane at a point (z_{0}, t_{0}) ∈ ∂Ω with z_{0} ∈ ω is given by equation (4.3), and b(z) is given by equation (4.4). The equation for the tangent can also be written as t = α(z − b(z_{0})). If Ω is lineally convex, then this tangent cannot intersect Ω, so we must have |t| ⩾ R(z) whenever z, z_{0} ∈ ω. Thus

|t| = |α(z − b(z_{0}))| ⩾ R(z) for all z, z_{0} ∈ ω;

inserting the value of |α| = 2|R_{z}(z_{0})| = |h_{z}(z_{0})|/\sqrt{h(z_{0})} we obtain

|z − b(z_{0})| ⩾ \frac{R(z)}{2|R_{z}(z_{0})|} = \frac{\sqrt{h(z)h(z_{0})}}{|h_{z}(z_{0})|}.

We now let z, z_{0} vary in ω to get the desired conclusion.

The idea is to prove that the tangent condition is not only necessary as in Lemma 9.4, but also sufficient if ω is a disk, which we shall do in Proposition 9.5. We then proceed to prove that Ω does satisfy the tangent condition under our hypotheses if ω is a disk.

Proposition 9.5. Assume that h ∈ C^{2}(\overline{D}), h > 0, satisfies the differential condition (9.1) and that Ω satisfies the tangent condition. Let φ be the function constructed in Example 9.2. Then φ(z\overline{z})h(z) satisfies the differential condition if s_{0} is sufficiently close to 1. Therefore, by Theorem 7.6, the open set \{(z, t) ∈ D×C; |t|^{2} < φ(z\overline{z})h(z)\}, which has a C^{2} boundary, is lineally convex; as a consequence also its limit as s_{0} tends to 1, viz. Ω itself, is lineally convex.

Proof. Using formula (4.4) for b(z) the relation between the inequality (9.2) used in the proof of Proposition 9.1 and the tangent condition is easy to establish. We observe that |b(z)| = |z − h(z)/h_{z}(z)| > |z| if and only if Re 2zh_{z}(z)/h(z) < 1. Thus if Ω satisfies the tangent condition, then h satisfies (9.2) for some a < 1 and all z in some sufficiently narrow annulus \sqrt{s_{0}} ⩽ |z| ⩽ 1.^{1}

Define

A = sup_{|z|⩽1} |\frac{2zh_{z}(z)}{h(z)}| and a(s_{0}) = sup_{\sqrt{s_{0}}⩽|z|⩽1} Re [\frac{2zh_{z}(z)}{h(z)}] .

If A ⩽ 1 we are done; otherwise we can choose s_{0} < 1 so close to 1 that (1 − s_{0})^{2} ⩽ 2(1 − a(s_{0}))/(A^{2} − 1). Proposition 9.1 can be applied and shows that φ(z\overline{z})h(z) satisfies the differential condition.

We shall now prove that it can never happen that Re 2zh_{z}(z)/h(z) ⩾ 1 for any z with |z| ⩽ 1.

Proposition 9.6. If h ∈ C^{2}(\overline{D}), h > 0, satisfies the differential condition (9.1), then Ω satisfies the tangent condition.

Proof. Let us define

b_{0}(r) = inf_{|z|⩽r} |b(z)|, 0 < r ⩽ 1.

This is a decreasing function and it is continuous where it is finite. The tangent condition for Ω_{r} = {(z, t) ∈ D(0, r) × C; |t|^{2} < h(z)} means precisely that b_{0}(r) > r. It is clear that the condition is satisfied for a very small r. Indeed, b(0) = −h(0)/h_{z}(0) is either ∞ or a non-zero complex number; in view of the continuity, |b(z)| > r if |z| ⩽ r and r is small enough.

If the tangent condition is satisfied for a particular Ω_{r}, then by Proposition 9.5 the set Ω_{r} is lineally convex, so Lemma 9.4 can be applied and shows that b_{0}(r) ⩾ r+ε, where ε = (inf_{|z|⩽1} R)/(2 sup_{|z|⩽1} |R_{z}|) > 0. We know that b_{0}(r) > r for small values of r, and we have just seen that if b_{0}(r) > r, then also b_{0}(r) ⩾ r + ε, for a positive ε which does not depend on r. Therefore that function cannot assume any value in the interval ]r, r + ε[: it must satisfy b_{0}(r) > r all the way up to and including r = 1. This means that Ω satisfies the tangent condition.

Theorem 9.7. Let h ∈ C^{2}(D), h > 0, satisfy the differential condition (9.1). Then the open set Ω = {(z, t) ∈ D × C; |t|^{2} < h(z)} is lineally convex.

1Here we could remark that it would be enough to require that b(z) ∉ \overline{ω} only for all z ∈ ∂ω, supposing that h ∈ C^{2}(\overline{ω}). The stronger condition used in Definition 9.3 is however easier to handle in the proof of Proposition 9.6.

Proof. If h ∈ C^{2}(\overline{D}) with h > 0 in \overline{D} we see from Proposition 9.6 that Ω satisfies the tangent condition, so that Proposition 9.5 can be applied. In the general case with h ∈ C^{2}(D), h > 0, we apply this result to a smaller disk rD, r < 1, to conclude that the domain over rD is lineally convex. Then we let r → 1.

References

Hörmander, Lars

1994 Notions of convexity. viii + 414 pp. Boston: Birkhäuser.

Kiselman, Christer O. ms Duality of functions defined in lineally convex set. Manuscript 1995, 25 pages, Uppsala University.

Zinov′ev, B. S. 1971 Analitičeskie uslovija i nekotorye voprosy approksimacii linejno-vypuklyh oblastej s gladkami granicami v prostranstve C^{n}. Izv. vuzov. Matematika 11, No. 6, 61–69.

Author’s address: Uppsala University, Department of Mathematics, P. O. Box 480, SE-751 06 Uppsala, Sweden.