Mixing properties of commuting nilmanifold automorphisms

c

2015 by Institut Mittag-Leffler. All rights reserved

Mixing properties of commuting nilmanifold automorphisms

by

Alexander Gorodnik

University of Bristol Bristol, U.K.

Ralf Spatzier

University of Michigan Ann Arbor, MI, U.S.A.

1. Introduction

Given a measure-preserving action of a (discrete) group Γ on a probability space (X, µ), we say that this action is (s+1)-mixing if for everyf0, ..., fs∈L^∞(X) andγ0, ..., γs∈Γ,

Z

X

^s Y

i=0

fi(γix)

dµ(x)−!

s

Y

i=0

Z

X

fidµ (1.1)

as γi₁γ_i⁻¹

2 !∞ for all i16=i2. In particular, 2-mixing corresponds to the usual notion of mixing. It was discovered by Ledrappier [13] that 2-mixing does not imply 3-mixing forZ²-actions. In this paper we will be interested in mixing of higher order for group actions. Mixing of all orders is a very widespread phenomenon for 1-parameter actions.

In particular, it is known to hold for many transformations satisfying some hyperbolicity assumptions. However, this is a measurable property that might arise for a multitude of other reasons which are not well understood. For instance, the horocyclic flow provides an example of a parabolic dynamical system which is mixing of all orders. A well-known longstanding question of Rokhlin asks whether mixing of order 2 implies mixing of all orders for a general measure-preserving transformation.

Very little is known about higher-order mixing for actions of large groups. We are only aware of two general families of actions of large groups on manifolds where the multiple mixing has been established—Z^l-actions by automorphisms on compact abelian groups and actions of simple Lie groups. K. Schmidt and Ward [22] proved that 2- mixingZ^l-action by automorphisms on compact connected abelian groups are mixing of

A. G. was supported in part by EPSRC grant EP/H000091/1 and ERC grant 239606. R. S. was supported in part by NSF grant DMS-0906085.

all orders, and Mozes [16] established mixing of all orders for ergodic actions of connected semisimple Lie groups with finite centre.

In this paper we investigate mixing properties of Z^l-actions by automorphisms on compact nilmanifolds. We prove that for such actions, 2-mixing implies mixing of all orders and establish quantitative estimates for 2-mixing and 3-mixing.

1.1. Main results

LetGbe a simply connected nilpotent group and Λ be a discrete cocompact subgroup.

We call the spaceX=G/Λ a compact nilmanifold. We denote by Aut(X) the group of continuous automorphisms αofG such thatα(Λ)=Λ. Then Aut(X) naturally acts on X and preserves the Haar probability measureµonX.

Our first main result concerns exponential 3-mixing. In order to obtain any quanti- tative estimate in (1.1), it is necessary to work in a class of sufficiently regular functions.

We denote byC^θ(X) the space of H¨older functions with exponentθ, defined with respect to a Riemannian metric onX.

Theorem 1.1. Let α:Z^l!Aut(X) be an action on a compact nilmanifold X such that every α(z), z6=0, is ergodic. Then there exists η=η(θ)>0 such that for every f0, f1, f2∈C^θ(X)and z0, z1, z2∈Z^l,

Z

X

f₀(α(z₀)x)f₁(α(z₁)x)f₂(α(z₂)x)dµ(x)

= Z

X

f0dµ Z

X

f1dµ Z

X

f2dµ

+O(N(z0, z1, z2)^−ηkf0k_Cθkf1k_Cθkf2k_Cθ), where N(z₀, z₁, z₂)=exp(min_i6=jkz_i−z_jk).

We note that this result is new even for the case of toral automorphisms. Previously, quantitative 2-mixing was established for toral automorphisms in [14] and for automor- phisms of more general compact abelian groups in [15]. Mixing of all orders for ergodic commuting toral automorphisms was established in [22]. The argument in [22] relies on finiteness of the number of non-degenerate solutions of S-unit equations established in [19]. Although there are explicit estimates on the number of such solutions, these esti- mates are not sufficient to derive any quantitative estimate for 3-mixing because it is also essential to know how the sets of solutions depend on the coefficients. In order to prove Theorem1.1, we use more delicate Diophantine estimates for linear forms in logarithms of algebraic numbers established in [24] (cf. Proposition2.2below).

We also prove mixing of all orders.

Theorem 1.2. Let α:Z^l!Aut(X) be an action on a compact nilmanifold X such that every α(z),z6=0,is ergodic. Then,for every f0, ..., fs∈L^∞(X)and z0, ..., zs∈Z^l,

Z

X

^s Y

i=0

f_i(α(z_i)x)

dµ(x) =

s

Y

i=0

Z

X

f_idµ+o(1)

as mini6=jkzi−zjk!∞. Moreover, the convergence is uniform over families of H¨older functions f0, ..., fs such that kf0k_Cθ, ...,kfsk_Cθ1.

This theorem extends the main result of [22] to general nilmanifolds. The proof in [22] utilises abelian Fourier analysis and properties of solutions of S-unit equations.

Our approach is based on the study of distribution of images of polynomial maps inX.

This reduces the proof to the investigation of certain Diophantine inequalities which are analysed using W. Schmidt’s subspace theorem. In order to prove an effective version of Theorem 1.2, one would need to estimate the size of non-degenerate solutions of these Diophantine inequalities in terms of complexities of coefficients (cf. Proposition3.1 below). However, this seems to be far out of reach of available techniques whens>2.

Finally, we discuss the problem of exponential mixing for shapes in Aut(X). This notion was introduced by K. Schmidt in [20] in order to better understand Ledrappier’s examples [13] which are not mixing of higher order. Ashape in Aut(X) is a collection of elementsα₀, ..., α_s∈Aut(X). We say that the shape is mixing if, for everyf₀, ..., f_s∈ L^∞(X),

Z

X

^s Y

i=0

fi(αⁿ_ix)

dµ(x)−!

s

Y

i=0

Z

X

fidµ

as n!∞. This property has been extensively studied in the context of commuting automorphisms of compact abelian groups (see, for instance, [6], [21, Chapter VIII], [26], and [27]).

We establish quantitative mixing for commuting Anosov shapes. We say that the shapeα0, ..., αs isAnosov ifαi₁α⁻¹_i

2 is an Anosov map for all i16=i2.

Theorem 1.3. Let X be a compact nilmanifold and α0, ..., αs∈Aut(X) be a com- muting Anosov shape. Then there exists %=%(θ)∈(0,1) such that, for every f0, ..., fs∈ C^θ(X) and n∈N,

Z

X

^s Y

i=0

fi(αⁿ_ix)

dµ(x) =

s

Y

i=0

Z

X

fidµ+O

%ⁿ

s

Y

i=0

kfik_Cθ

. (1.2)

1.2. Applications to rigidity

The exponential mixing property played an important role in the program of classification of smooth Anosov higher-rankZ^l-actions on compact manifolds. It is expected that all

such actions can be built from actions by automorphisms on nilmanifolds. Fisher, Kalinin and Spatzier in [8] applied the exponential 2-mixing property and regularity results from [17] to extend their results for AnosovZ^l-actions on tori to actions on nilmanifolds.

Theorem 1.4. (Fisher, Kalinin, and Spatzier) Let αbe a C^∞-action of Z^l, l>2, on a compact nilmanifold X and let %:Z^l!Aut(X)be the map induced by the action of α(Z^l)on the fundamental group of X. Assume that there is a Z² subgroup of Z^l such that %(z) is ergodic for every non-zero z∈Z², and there is an Anosov element for α in each Weyl chamber of %. Then αis C^∞-isomorphic to %.

In fact, this application to global rigidity was our original motivation to establish the exponential mixing property for nilmanifold automorphisms.

Recently, Rodriguez Hertz and Wang [18] generalised Theorem 1.4and established a global rigidity result using only existence of a single Anosov element. Again, they crucially use the exponential mixing property, and reduce the problem to the prior result by showing existence of many Anosov elements.

We also use the exponential mixing property to establish cocycle rigidity for higher- rankZ^l-actions by automorphisms of nilmanifolds, extending the results of Katok and Spatzier [11], [12]. A C^∞-cocycle is aC^∞-mapc:Z^l×X!Rsatisfying the identity

c(z₁+z₂, x) =c(z₁, z₂x)+c(z₂, x) forz₁, z₂∈Z^land x∈X.

Two cocyclesc1 andc2 are calledsmoothly cohomologous if there exists b∈C^∞(X) such that

c1(z, x) =c2(z, x)+b(zx)−b(x) forz∈Z^landx∈X.

We call a cocycle constant if it does not depend onx∈X. We prove that cocycles over genuine higher-rank actions by automorphisms on nilmanifolds are smoothly cohomolo- gous to constant cocycles. This phenomenon was first discovered by Katok and Spatzier in [11] for certain higher-rank Anosov actions. Using our methods, we generalise this cocycle rigidity theorem to actions by automorphisms on nilmanifolds. We emphasise that the action in the following theorem need not be Anosov.

Theorem 1.5. Let α:Z^l!Aut(X) be an action on a compact nilmanifold X. As- sume that there is a Z²subgroup of Z^lsuch that α(z)is ergodic for every non-zeroz∈Z². Then every smooth R-valued cocycle is smoothly cohomologous to a constant cocyle.

For certain actions by partially hyperbolic left translations on homogeneous spaces G/Γ, where Gis a semisimple Lie group and Γ is a lattice inG, a similar theorem was proved by Damjanovic and Katok [3]–[5] and Wang [25]. We note that these authors also

prove H¨older versions of this result which are not amenable to our techniques. Further- more, cocycle rigidity results are proven for small perturbations of these actions onG/Γ in [5] and [25]. Again we cannot obtain these results by our methods.

Acknowledgements

A. G. would like to thank the University of Michigan for hospitality during his visit when the work on this project had started. R. S. thanks the University of Bristol for hospitality and support during this work.

2. Exponential 3-mixing

In this section we prove Theorem 1.1. We start by setting up basic notation, which will be also used in subsequent sections. Then, in§§2.2–2.4, we collect some auxiliary estimates. The proof of Theorem 1.1 is divided into two parts. We first give a proof under an irreducibility condition in§2.5, and then in§2.6prove Theorem1.1 in general using an inductive argument.

We note that if the reader is only interested in exponential 2-mixing, then the results of§2.3 are not needed, and in§2.5, one only needs to consider case 1. This makes the proof much simpler.

2.1. Notation

LetGbe a connected simply connected nilpotent Lie group, Λ be a discrete cocompact subgroup, andX=G/Λ be the corresponding nilmanifold equipped with the invariant probability measureµ. We fix a right-invariant Riemannian metric d onG which also defines a Riemannian metric onX. LetL(G) be the Lie algebra ofGand exp:L(G)!G be the exponential map. The lattice subgroup Λ defines a rational structure on L(G).

For a fieldK⊃Q, we denote byL(G)K the corresponding Lie algebra overK. Denoting the commutator subgroup byG⁰, let π:G!G/G⁰ be the factor map. We also have the corresponding mapDπ:L(G)!L(G/G⁰). We fix an identificationG/G⁰'L(G/G⁰)'R^d that respects the rational structures.

Every automorphism β ofGdefines a Lie-algebra automorphism Dβ:L(G)!L(G) such thatβexp=expDβ. Ifβ(Λ)=Λ, thenDβpreserves the rational structure ofL(G) defined by Λ. In particular, given an actionα:Z^l!Aut(X) on the nilmanifoldX=G/Λ, we obtain a homomorphismDα:Z^l!GL(L(G)_Q).

For a multiplicative characterχ:Z^l!C^×, we set

Lχ:={u∈ L(G)⊗C:Dα(z)u=χ(z)uforz∈Z^l}.

Let X(α) denote the set of characters χ appearing in the action Dα on L(G), and X⁰(α)⊂X(α) be the set of characters appearing in the action onL(G)/L(G)⁰.

2.2. Estimates on Lyapunov exponents

Sinceα(Z^l) preserves the rational structure on L(G) defined by the lattice Λ, it follows that each characterχinX(α) is of the formχ(z)=u^z₁¹... u^z_l^l where theui’s are algebraic numbers. The Galois group Gal(Q/Q) naturally acts onX(α) andX⁰(α). LetX0⊂X⁰(α) be one of the Galois orbits.

Lemma2.1. Suppose that everyα(z),z6=0,acts ergodically onX. Then there exists c>0such that

χ∈Xmax0

|χ(z)|>exp(ckzk) for all z∈Z^l.

Proof. By [2, Theorem 5.4.13], ΛG⁰/G⁰ is a lattice inG/G⁰, and the actionαdefines the action on the torusT:=G/ΛG⁰ by linear automorphisms. LetV be the subspace of L(G)/L(G)⁰spanned by theχ-eigenspaces withχ∈X0. Clearly, this subspace is invariant under α(Z^l) and is defined over Q. Hence, it defines an α-invariant subtorusT_X0 of T. Sinceα(z)|T is ergodic when z6=0, it follows that the corresponding linear map has no roots of unity as eigenvalues. This implies thatα(z)|TX0 is also ergodic.

Consider a linear map`:R^l!R^|X0|which is defined forz∈Z^lby

`(z) := (log|χ(z)|:χ∈ X0)

and extended toR^lby linearity. Since for everyz∈Z^l\{0}, the automorphismα(z) acting onT_X0 is ergodic, we have`(z)6=0 by [9, Lemma 3.2]. Hence,`|_Zl is injective.

We also claim that`(Z^l) is discrete. We consider the embeddingZ^l!GL(V) defined byα. Sinceα(Z^l) preserves the integral lattice in V corresponding to the torusTX₀, it follows that the image of this embedding is discrete. In other words, the subset

{(χ(z) :χ∈ X0) :z∈Z^l}

of (C^×)^|X0| is discrete. Since the kernel of the natural homomorphism (C^×)^|X0|!R^|X0| defined by s7!log|s| is compact, this implies that `(Z^l) is discrete, as claimed. Since

`(Z<sup>l</sup>) is discrete and has rankl, it follows that the space`(R^l) has dimensionl, and, in

particular, the map`is injective. Therefore, by compactness, there existsc>0 such that, for everyz∈R^l, we have

χ∈Xmax0

log|χ(z)|>ckzk.

This implies the lemma.

Lemma 2.1shows that in Theorem1.1we may replaceN(z₀, z₁, z₂) by exp

min

i6=j max

χ∈X0

|χ(zi−zj)|

.

2.3. Diophantine estimates

Recall that the (absolute)height of an algebraic number uis defined by

H(u) =

Y

v

max{1,|u|v}

1/[Q(u):Q]

,

where | · |v denote suitably normalised absolute values of the field Q(u). When uis an algebraic integer, the height can be computed as

H(u) =

Y

i

max{1,|ui|}

1/[Q(u):Q]

,

where theui’s denote all the Galois conjugates ofu.

The following result is deduced from the work of Waldschmidt [24, Corollary 10.1].

Proposition 2.2. Let u₁, ... u_l, u∈C be algebraic numbers and z=(z₁, ..., z_l)∈Z^l. Then there exist c₁, c₂, c₃>1,depending on u₁, ..., u_l and [Q(u):Q], such that, assuming that

kzk>log(c2H(u)) (2.1)

and

u^z₁¹... u^z_l^lu6= 1, we have the estimate

|u^z₁¹... u^z_l^lu−1|>exp

−c1log(c₂H(u)) log

c₃kzk log(c2H(u))

. (2.2)

Surprisingly, it turns out that the term log(c₂H(u)) in the denominator is essential to establish exponential 3-mixing (cf. (2.28)–(2.30) below).

Proof. We note that, sinceH(u)>1 and (2.1) holds, the right-hand side of (2.2) is bounded from above by

exp(−c1logc2logc3).

Taking the constants sufficiently large, we may arrange that this quantity is bounded by ¹₂. Then (2.2) trivially holds when|u^z₁¹... u^z_l^lu−1|>¹2, and without loss of generality we assume that|u^z₁¹... u^z_l^lu−1|6¹2.

Let log denote the principle value of the (complex) logarithm. There exists z₀∈Z such that|z0|kzkand

T:= log(u^z₁¹... u^z_nⁿu) =πiz0+z1logu1+...+zllogul+logu.

It is convenient to set u0=−1, so that logu0=πi. (Here, but not elsewhere, i denotes the imaginary unit.) LetS:=u^z₁¹... u^z_l^lu. Since|S−1|6¹2,

|T|=|logS|62|S−1|.

Hence, it is sufficient to establish a lower bound for|T|. Note that, sinceS6=1, we have T6=0. For this purpose, we use [24, Corollary 10.1], which we now recall. We note that the result in [24] is stated using the logarithmic height while here we use the exponential height. For simplicity, we takeE=eandf=1.

LetD=[Q(u0, ..., ul, u):Q],A0, ..., Aland B be numbers, greater thane, such that

H(u_i)6A_i, i= 0, ..., l, H(u)6B and

l

X

i=0

|logu_i| logAi

+|logu|

logB 6e⁻¹(l+2)D. (2.3) We set

A= max{A0, ..., Al, B}, M= max

i=0,...,l

1 logAi

+ |zi| logB

, Z0= max{7+3 log(l+2),logD}, G₀= max{4(l+2)Z₀,logM,logD},

U₀= max{D²logA, D^l+4G₀Z₀logA₀...logA_llogB}.

Then, according to [24, Corollary 10.1],

|T|>exp(−cU0), (2.4)

wherec is an explicit positive constant depending only onn. We setB:=c2H(u) with c2>1. We note that

|logu|²6π²+(log|u|)²6π²+[Q(u) :Q]²(logH(u))².

Therefore, takingAisufficiently large, depending onui, and sufficiently largec2, we may arrange that (2.3) holds. If c2 is sufficiently large,A=B. Under the assumption (2.1), we haveM6c3kzk/logB with sufficiently large c3 and also

G₀= logM6log c3kzk

logB

. Moreover, ifc3is sufficiently large, then

U₀log c₃kzk

logB

logB.

Therefore, estimate (2.4) implies that

|T|>exp

−c₁log c3kzk

logB

logB

,

wherec1is an explicit positive constant. This completes the proof of the proposition.

2.4. Equidistribution of box maps Abox mapis an affine map

ι:B:= [0, T₁]×...×[0, Tk]−!L(G) of the form

ι: (t1, ..., tk)7−!v+t1w1+...+tkwk, (2.5) with v, w1, ..., wk∈L(G). We shall use the following result, which is a variation of our theorem [9, Theorem 2.1], that implies equidistribution of box maps under suitable Dio- phantine conditions. This result is based on the work of Green and Tao [10].

We denote by|B|thek-dimensional volume of the box B.

Theorem 2.3. Fix 0<θ61. There exist L1, L2>0 such that for every δ∈(0, δ0) and every box map ι:B!L(G), one of the following conditions holds:

(i) For everyθ-H¨older function f:X!R, u∈L(G),andg∈G,

1

|B|

Z

B

f(exp(u) exp(ι(t))gΛ)dt−

Z

X

f dµ

6δkfk_Cθ. (ii) There exists z∈Z^d\{0} such that

kzk δ^−L1 and |hz, Dπ(wi)i| δ^−L2 Ti

for all i= 1, ..., k.

Proof. In the case of Lipschitz functionsf, this is [9, Theorem 2.1], and the analo- gous result for H¨older functions can be deduced by a standard approximation argument.

Indeed, suppose that for somef∈C^θ(X),u∈L(G), andg∈G,

1

|B|

Z

B

f(exp(u) exp(ι(t))gΛ)dt−

Z

X

f dµ

> δkfk_Cθ. (2.6) Then one can find a Lipschitz functionf_εsuch that

kfε−fk_C06ε^θkfk_Cθ and kfεkLipε^{−dim(X)−1}kfk_C0

(see, for instance, [9, Lemma 2.4]). Then takingε= ¹₃δ^1/θ

, we deduce from (2.6) that

1

|B|

Z

B

fε(exp(u) exp(ι(t))gΛ)dt−

Z

X

fεdµ

>(δ−2ε^θ)kfk_Cθ

ε^dim(X)+1(δ−2ε^θ)kf_εk_Lip δ(dim(X)+1)/θ+1kf_εk_Lip.

Now the theorem for Lipschitz functions implies that (ii) holds with some L1, L2>0 depending onθ.

2.5. 3-mixing under an irreducibility condition

The action ofDα(Z^l) preserves the rational structure on L(G) defined by the lattice Λ.

In particular, it follows that each character χ in X(α) is of the form χ(z)=u^z₁¹... u^z_l^l where the ui’s are algebraic numbers. The Galois group Gal(Q/Q) naturally acts on X(α) and onL(G)_Q. We fix an orbitX0⊂X⁰(α) of the Galois group and for eachχ∈X0, we fix a vectorwχ∈Lχ whose coordinates are algebraic integers, so that the vectorswχ

are also conjugate under the action of the Galois group. LetW_C be the Lie subalgebra ofL(G)⊗Cgenerated by the vectorswχ,χ∈X0, andW=W_C∩L(G). We also fix a basis {wi}i ofW.

In this section, we prove Theorem 1.1 under the irreducibility assumption that Dπ(W) is not contained in a proper rational subspace. Let wχ=Dπ(wχ), χ∈X0. We observe that under this assumption the coordinates of each of the vectorswχ are linearly independent over Q. Indeed, if we suppose that ha,wχi=0 for some a∈Q^d\{0}, then applying the action of the Galois group, we deduce thatha,wχi=0 for all χ∈X0. Since Dπ(W) is spanned overC by the vectors wχ, χ∈X0, this would imply that Dπ(W) is contained in a proper rational subspace, which contradicts our assumption.

Let

N=N(z1, z2, z3) := exp

mini6=j kzi−zjk

. (2.7)

Without loss of generality, we may assume that z0=0 and N=exp(kz1−z2k). We set ε=N⁻, where>0 is a fixed parameter which is sufficiently small and will be specified later (see (2.21), (2.23), (2.26) and (2.30) below).

We fix a fundamental domain F⊂Gfor X=G/Λ and set E=exp⁻¹(F). As in [9,

§3], we may arrange thatE is bounded and has piecewise smooth boundary. Since the Haar measure onGis the image under exp of a suitably normalised Lebesgue measure onL(G) [2, Theorem 1.2.10], we obtain

Z

X

f₀(x)f₁(α(z₁)x)f₂(α(z₂)x)dµ(x)

= Z

E

f₀(exp(u)Λ)f₁(exp(Dα(z₁)u)Λ)f₂(exp(Dα(z₂)u)Λ)du.

(2.8)

We choose a basis of L(G) that contains the basis {w_i}_i of W and tessellate L(G) by cubesCof sizeεwith respect to this basis. SinceE has piecewise smooth boundary, we obtain

E\ [

C⊂E

C

ε, (2.9)

and Z

E

f₀(exp(u)Λ)f₁(exp(Dα(z₁)u)Λ)f₂(exp(Dα(z₂)u)Λ)du

=X

C⊂E

Z

C

f₀(exp(u)Λ)f₁(exp(Dα(z₁)u)Λ)f₂(exp(Dα(z₂)u)Λ)du +O(εkf0k_Cθkf1k_Cθkf2k_Cθ).

(2.10)

For every cubeC, we pick a pointuC∈C. Then, sincef0 isθ-H¨older, Z

C

f0(exp(u)Λ)f1(exp(Dα(z1)u)Λ)f2(exp(Dα(z2)u)Λ)du

=f0(exp(uC)Λ) Z

C

f1(exp(Dα(z1)u)Λ)f2(exp(Dα(z2)u)Λ)du +O(ε^θkf0k_Cθkf1k_Cθkf2k_Cθ).

(2.11)

We decompose each cubeC asC=B⁰+B, whereB is a cube inW and B⁰ is a cube in the complementary subspace.

We claim that, for sufficiently small>0 and all sufficiently largeN defined in (2.7), 1

|B|

Z

B

f1(exp(v1+Dα(z1)b)Λ)f2(exp(v2+Dα(z2)b)Λ)db

= Z

X

f1dµ Z

X

f2dµ

+O(N⁻kf1k_Cθkf2k_Cθ),

(2.12)

uniformly over the cubesB andv1, v2∈L(G).

Suppose first that (2.12) holds. Then using uniformity over v1 and v2, we deduce that

1

|C|

Z

C

f1(exp(Dα(z1)u)Λ)f2(exp(Dα(z2)u)Λ)du

= 1

|B⁰| |B|

Z

B⁰

Z

B

f1(exp(Dα(z1)b⁰+Dα(z1)b)Λ)f2(exp(Dα(z2)b⁰+Dα(z2)b)Λ)db db⁰

= Z

X

f1dµ Z

X

f2dµ

+O(N⁻kf1k_Cθkf2k_Cθ).

Combining this estimate with (2.10) and (2.11), we obtain Z

E

f₀(exp(u)Λ)f₁(exp(Dα(z₁)u)Λ)f₂(exp(Dα(z₂)u)Λ)du

=

X

C⊂E

f0(exp(uC)Λ)|C|

Z

X

f1dµ Z

X

f2dµ

+O((N⁻+ε^θ)kf0k_Cθkf1k_Cθkf2k_Cθ).

Sincef isθ-H¨older and (2.9) holds, X

C⊂E

f₀(exp(u_C)Λ)|C|=X

C⊂E

Z

C

f₀(exp(u)Λ)du+O(ε^θkf₀k_Cθ)

= Z

E

f0(exp(u)Λ)du+O((ε+ε^θ)kf0k_Cθ)

= Z

X

f0dµ+O(ε^θkf0k_Cθ).

Hence, Z

E

f0(exp(u)Λ)f1(exp(Dα(z1)u)Λ)f2(exp(Dα(z2)u)Λ)du

= Z

X

f₀dµ Z

X

f₁dµ Z

X

f₂dµ

+O(N^−θkf₀k_Cθkf₁k_Cθkf₂k_Cθ).

(2.13)

This proves the required estimate whenN is sufficiently large, and it is also clear that this estimate holds for N in bounded intervals. Hence, Theorem 1.1 follows. Now it remains to prove the claim (2.12).

To prove (2.12), we apply Theorem 2.3to the nilmanifold X×X=(G×G)/(Λ×Λ) withδ=N⁻. We assume thatN is sufficiently large, so that Theorem2.3applies. Let f=f1⊗f2. Clearly,

Z

X×X

f d(µ⊗µ) = Z

X

f1dµ Z

X

f1dµ

and kfk_Cθ kf1k_Cθkf2k_Cθ.

We consider the mapι: [0, ε]^k!L(G) defined by

ι(t) =

v⁰₁+

k

X

i=1

t_iDα(z₁)w_i, v⁰₂+

k

X

i=1

t_iDα(z₂)w_i

,

with suitably chosenv⁰₁, v₂⁰∈L(G), so that Z

B

f₁(exp(v₁+Dα(z₁)b)Λ)f₂(exp(v₂+Dα(z₂)b)Λ)db= Z

[0,ε]^k

f(ι(t)Λ)dt.

It is sufficient to prove that ε^−k

Z

[0,ε]^k

f(ι(t)Λ)dt= Z

X×X

f d(µ⊗µ)+O(δkfk_Cθ).

Applying Theorem2.3, we deduce that either

ε^−k Z

[0,ε]^k

f(ι(t)Λ)dt−

Z

X×X

f d(µ⊗µ)

6δkfk_Cθ, (2.14) or there exists (a1, a2)∈(Z^d)²\{(0,0)}such that

max{ka1k,ka2k} δ^−L1=N^L1 (2.15) and

|ha1,(Dπ)Dα(z₁)w_ii+ha2,(Dπ)Dα(z₂)w_ii| δ^−L2

ε =N^(L2+1) (2.16) for alli=1, ..., k.

We shall show that if>0 is sufficiently small andNis sufficiently large, then either (2.15) or (2.16) fails. Suppose that both (2.15) and (2.16) holds. Since each of the vectors wχ, χ∈X0, is a linear combination of vectorswi, we deduce from (2.16) that

|ha1,(Dπ)Dα(z1)wχi+ha2,(Dπ)Dα(z2)wχi| N^(L2+1) for allχ∈ X0. (2.17) AsDα(z)w_χ=χ(z)w_χ andw_χ=Dπ(w_χ), (2.17) becomes

|χ(z1)ha1,wχi+χ(z2)ha2,wχi| N^(L2+1) for allχ∈ X0. (2.18) We divide the argument into three cases.

Case 1. a1=0. Thena26=0 andha2,wχi6=0. Moreover, by [1, Theorem 7.3.2],

|ha2,w_χi| ka2k^−d−1N^−L1(d+1). (2.19)

By Lemma2.1, there existsχ∈X0such that|χ(z2)|>N^cwith fixedc>0. Hence, it follows from (2.18) that

|ha2,wχi| N^(L2+1)−c. (2.20)

We assume that the parameter>0 satisfies

−L1(d+1)>(L2+1)−c. (2.21) Comparing (2.19) and (2.20), we get a contradiction ifN is sufficiently large. Hence, we may assume thata₁6=0.

Case 2. a16=0and χ(z1)ha1,wχi+χ(z2)ha2,wχi=0for some χ∈X0. As the Galois group acts transitively on the setX0, it follows that this equality holds for allχ∈X0. By Lemma2.1, there existsχ∈X0such that|χ(z2−z1)|>N^c with fixedc>0. Then

|ha₂,w_χi|=|χ(z₁−z₂)| |ha₁,w¯_χi|>N^c|ha₁,w_χi|. (2.22) Sincea16=0, we haveha1,wχi6=0, and by [1, Theorem 7.3.2],

|ha1,w_χi| ka1k^−d−1N^−L1(d+1). On the other hand,

|ha2,wχi| ka2k N^L1. Hence, we deduce that

N^−L1(d+1)+cN^L1. We choose the parameter>0 so that

−L₁(d+1)+c >L₁. (2.23)

Then whenN is sufficiently large, we get a contradiction.

Case 3. a₁6=0and χ(z₁)ha₁,w_χi+χ(z₂)ha₂,w_χi6=0for all χ∈X₀. This is the most difficult part of the proof.

Sincea16=0, we haveha1,wχi6=0, and, by [1, Theorem 7.3.2],

|ha1,wχi| ka1k^−d−1N^−L1(d+1).

We setu=−ha₂,w_χi/ha₁,w_χi. By Lemma2.1, there existsχ∈X₀such that|χ(z₁)|N^c with fixedc>0. It follows from (2.18) that for thisχ, we have the estimate

|χ(z2−z1)u−1| N^(L2+1)

|χ(z1)| |ha1,wχi|N^{(L2+1+L1(d+1))−c}. (2.24)

LetK1:=(L2+1+L1(d+1))−c.

Next, we compare this estimate with the lower estimate provided by Proposition2.2.

We note that

H(u) =Y

v

max{|ha1,w_χi|v,|ha2,w_χi|v}^1/[Q(u):Q].

For all non-Archimedian placesv,

|hai,wχi|v61, and for all Archimedianv,

|hai,wχi|v kaik N^L1. Therefore,

H(u)N^K2, (2.25)

whereK2:=L⁰₁ with fixedL⁰₁>0. We take the parameter>0 so that

K2=L⁰₁<1. (2.26)

Then assuming thatN is sufficiently large, we obtain

log(c2H(u))6log(c⁰₂N^K2)6logN, (2.27) wherec⁰₂>1 depends on the implicit constant in the estimate (2.25). We recall that we have chosen the indices so that

logN=kz₂−z₁k.

Since (2.27) holds, Proposition2.2 applies, and we deduce that

|χ(z2−z1)u−1|>exp

−c1log(c2H(u)) log

c₃kz2−z1k log(c2H(u))

.

Without loss of generality, we may assume thatc₃>e. Since the functionx7!xlog(C/x) is increasing forx6C/e, we deduce that

|χ(z2−z1)u−1|>exp

−c1log(c⁰₂N^K2) log

c3logN log(c⁰₂N^K2)

>exp(−c1log(c⁰₂N^K2) log(c₃K₂⁻¹)).

(2.28)

Comparing (2.24) and (2.28), we conclude that

K₂⁰ logN+M26K1logN+M1, (2.29) where K₂⁰:=−c1K2log(c3K₂⁻¹), M2:=−c1logc⁰₂log(c3K₂⁻¹), and M1 is determined by the implicit constant in (2.24). We observe that, as !0⁺, K₂⁰!0⁻ and K1!−c<0.

Therefore, taking the parameter>0 sufficiently small, we may arrange that

K₂⁰> K1. (2.30)

Then when N is sufficiently large, (2.29) fails. This shows that either (2.15) or (2.16) fails, and (2.14) holds whenN is sufficiently large. Now we have verified the claim (2.12) and completed the proof of Theorem1.1under the irreducibility condition.

In order to prove Theorem 1.1 in general, we observe that using the same argu- ment, one can deduce the following more general version of the estimate (2.12): for all sufficiently largeN defined in (2.7),

1

|B|

Z

B

f₁(h₁β₁(exp(v₁+Dα(z₁)b))Λ)f₂(h₂β₂(exp(v₂+Dα(z₂)b))Λ)db

= Z

X

f1dµ Z

X

f2dµ

+O(N⁻kf1k_Cθkf2k_Cθ)

(2.31)

uniformly over the cubesB,h1, h2∈G,v1, v2∈L(G), and automorphismsβ1andβ2ofG which act trivially onG/G⁰. Indeed,

Z

B

f1(h1β1(exp(v1+Dα(z1)b))Λ)f2(h2β2(exp(v2+Dα(z2)b))Λ)db

= Z

B

f1(h1exp(Dβ1(v1)+Dβ1Dα(z1)b)Λ)f2(h2exp(Dβ2(v2)+Dβ2Dα(z2)b)Λ)db, and to prove (2.31), we can apply Theorem2.3 to the map

ι:t7−! v⁰₁+

k

X

i=1

tiDβ1Dα(z1)wi, v⁰₂+

k

X

i=1

tiDβ2Dα(z2)wi

.

As in the above proof, either (2.31) holds, or an analogue of (2.16) holds, but since DπDβi=Dπ, this reduces to exactly the same estimate as (2.16). Therefore, (2.31) follows. Now we combine (2.31) with the argument (2.8)–(2.13) to deduce that

Z

X

f0(x)f1(h1β1(α(z1)(x)))f2(h2β2(α(z2)(x)))dµ(x)

= Z

X

f₀dµ Z

X

f₁dµ Z

X

f₂dµ

+O(N^−θkf₀k_Cθkf₁k_Cθkf₂k_Cθ)

(2.32)

uniformly over h₁, h₂∈G and automorphisms β₁ and β₂ of G that preserve Λ and act trivially onG/G⁰. We will use this estimate to establish Theorem1.1in general.

2.6. 3-mixing in general

LetW be the Lie subalgebra of L(G) introduced in§2.5. By [23, Chapter 5,§5], there exists a closed connected normal subgroupM ofGsuch thatM/(M∩Λ) is compact, and

exp(W)gΛ =M gΛ for almost everyg∈G.

Since we may replace the lattice Λ by its conjugate, we assume that exp(W)Λ =MΛ.

We note that the groupM satisfies the following properties:

(i) M isα(Z^l)-invariant;

(ii) Dπ(W) is not contained in a proper rational subspace ofL(M/M⁰);

(iii) [G, M]⊂M⁰.

Properties (i)–(iii) can be verified exactly as in the proof of [9, Lemma 3.4].

We give the proof of Theorem 1.1using induction on the dimension ofX. For this, we use thatX=G/Λ fibers overY=G/MΛ with fibers isomorphic to

R=MΛ/Λ'M/(M∩Λ).

The invariant measure onX can be decomposed as Z

X

f dµ= Z

Y

Z

R

f(yr)dµ_R(r)dµ_Y(y), f∈C(X),

whereµ_Y andµ_R are normalised invariant measure onY andR, respectively. Since the fibration isα(Z^l)-equivariant (by (i)),

Z

X

f0(x)f1(α(z1)x)f2(α(z2)x)dµ(x)

= Z

Y

Z

R

f0(yr)f1(α(z1)(y)α(z1)(r))f2(α(z2)(y)α(z2)(r))dµR(r)

dµY(y)

= Z

F

Z

R

f0(gr)f1(α(z1)(g)α(z1)(r))f2(α(z2)(g)α(z2)(r))dµR(r)

dmF(g), (2.33)

whereF⊂Gis a bounded fundamental domain forG/MΛ, andmF is the measure onF induced byµY. We shall show that forN defined in (2.7) and some η >0,

Z

R

f₀(gr)f₁(α(z₁)(g)α(z₁)(r))f₂(α(z₂)(g)α(z₂)(r))dµ_R(r)

= Z

R

f0(gr)dµR(r) Z

R

f1(α(z1)(g)r)dµR(r) Z

R

f2(α(z2)(g)r)dµR(r)

+O(N^−ηkf0k_Cθkf1k_Cθkf2k_Cθ)

(2.34)

uniformly overg∈F.

Suppose that (2.34) holds. Then, combining (2.33) and (2.34), we obtain Z

X

f₀(x)f₁(α(z₁)x)f₂(α(z₂)x)dµ(x)

= Z

Y

f¯₀(y) ¯f₁(α(z₁)y) ¯f₂(α(z₂)y)dµ_Y(y)+O(N^−ηkf₀k_Cθkf₁k_Cθkf₂k_Cθ), where the functions ¯fionY are defined by

y7−!Z

R

f_i(yr)dµ_R(r).

Since dim(Y)<dim(X), it follows from the inductive assumption that, for someη >0, Z

Y

f¯₀(y) ¯f₁(α(z₁)y) ¯f₂(α(z₂)y)dµ_Y(y)

= Z

Y

f¯₀dµ_Y Z

Y

f¯₁dµ_Y Z

Y

f¯₂dµ_Y

+O(N^−ηkf¯₀k_Cθkf¯₁k_Cθkf¯₂k_Cθ)

= Z

X

f₀dµ Z

X

f₁dµ Z

X

f₂dµ

+O(N^−ηkf₀k_Cθkf₁k_Cθkf₂k_Cθ), and this completes the proof of Theorem1.1. Hence, it remains to prove (2.34).

To prove (2.34), we write

α(zi)(g) =aimiλi withai∈F,mi∈M andλi∈Λ, i= 1,2.

Then Z

R

f0(gr)f1(α(z1)(g)α(z1)(r))f2(α(z2)(g)α(z2)(r))dµR(r)

= Z

R

f0(gr)f1(a1m1β1(α(z1)(r)))f2(a2m2β2(α(z2)(r)))dµR(r),

where the β_i’s are the maps induced by the automorphisms m7!λ_imλ⁻¹_i . We observe that because of (ii),W⊂L(M) satisfies the irreducibility assumption of§2.5, and by (iii), the automorphisms β_i act trivially on M/M⁰. Hence, (2.32) holds. We apply (2.32) to the functions onRdefined by

φ₀(r) :=f₀(gr) and φ_i(r) :=f_i(a_ir), i= 1,2.

This gives Z

R

φ₀(r)φ₁(m₁β₁(α(z₁)r))φ₂(m₂β₂(α(z₂)r))dµ_R(r)

= Z

R

φ0dµR

Z

R

φ1dµR

Z

R

φ2dµR

+O(N^−ηkφ0k_Cθkφ1k_Cθkφ2k_Cθ)

= Z

R

f₀(gr)dµ_R(r) Z

R

f₁(α(z₁)(g)r)dµ_R(r) Z

R

f₂(α(z₂)(g)r)dµ_R(r)

+O(N^−ηkf0k_Cθkf1k_Cθkf2k_Cθ).

This implies (2.34) and completes the proof of Theorem1.1.

3. Higher-order mixing

The aim of this section is to prove Theorem1.2. We shall use the notation introduced in §2.1. In §3.1 we prepare Diophantine estimates. Then in §3.2 we give a proof of Theorem 1.2 under an irreducibility condition, and in§3.3 we give a proof in general using an inductive argument.

We note that it is sufficient to prove Theorem 1.2 for a collection of functions f_i∈L^∞(X) which is dense in L¹(X). Hence, we may assume that f₀, ..., f_s∈C^θ(X).

Furthermore, we may assume thatz₀=0.

3.1. Diophantine estimates

LetKbe a number field andSbe a finite set of places ofKcontaining all the archimedean places. We denote byU_S the ring ofS-units, namely, the group of elementsxinK such that|x|v=1 forv /∈S. For a vector ¯x∈K^s, we define its (relative) height by

H(¯x) =Y

v

max{1,kxk¯ v},

wherev runs the set of all places ofK, andk¯xkv=maxi|xi|v.

Proposition 3.1. Let v∈Sand b1, ... bs∈K\{0}. Then for every ε>0,the inequal- ity

b1+

s

X

j=2

bjxj

_v

< H(¯x)^−ε (3.1)

has finitely many solutionsx∈U¯ S such that no proper subsum ofb1+Ps

j=2bjxj vanishes.

We call such solutions of (3.1)non-degenerate.

We give a simple proof of the proposition which is based on the classical W. Schmidt subspace theorem. We note that this proposition is closely related to results about finiteness of the number of solutions ofS-unit equations. ForS-unit equations the number of non-degenerate solutions can be estimated explicitly. For instance, we refer to a remarkably uniform bound in [7]. Since explicit bounds on the number of solutions do not play any role in our arguments, we do not pursue this direction here.

Proof. We prove the proposition by induction on s. Note that, when s=1, the statement holds trivially because there are only finitely many solutions ofH(¯x)<c.

Given a solution ¯x of (3.1), we set ¯y=(1,x), and we denote by¯ jv=jv(¯x) the first indexjv such that

|yjv|v=kyk¯ v>1.

Partitioning the set of solutions according to the indexjv, we may assume that this index is fixed.

We introduce a family of linear formsLwj(¯y), withw∈S andj=1, ..., s, defined by

L_wj(¯y) =











y_j, if (w, j)6= (v, j_v),

s

X

j=1

bjyj, if (w, j) = (v, jv).

Then, if ¯y=(1,x) corresponds to a solution of (3.1),¯

s

Y

j=1

|Lwj(¯y)|w=

s

Y

j=1

|yj|w, w6=v,

s

Y

j=1

|Lvj(¯y)|v=|Lvj_v(¯y)|v

Y

j6=jv

|yj|v< H(¯y)^−ε

s

Y

j=1

|yj|v,

and, by the product formula, Y

w∈S s

Y

j=1

|Lwj(¯y)|w< H(¯y)^−ε. (3.2) By the W. Schmidt subspace theorem [1, Corollary 7.2.5], all the solutions of (3.2) are contained in a finite union of proper linear subspaces ofK^s. Partitioning solutions of (3.1) according to these subspaces, we may assume that these solutions additionally satisfy a non-trivial linear relation

c₁+

s

X

j=2

c_jx_j= 0 (3.3)

withc₁, ..., c_s∈K.

Suppose thatc₁6=0. Given a solution ¯xof (3.3), we pick a minimalJ⊂{2, ..., s}such that

c₁+X

j∈J

c_jx_j= 0. (3.4)

Then no proper subsum in (3.4) vanishes. It follows from the finiteness of the number of non-degenerate solutions of unit equations [1, Theorem 7.4.2] thatxj,j∈J, varies over a finite set. This shows that for every solution ¯xof (3.3) there exists j0=2, ..., ssuch that xj₀ belongs to a fixed finite set. Hence, in order to prove finiteness of non-degenerate solutions (3.1), we may assume that xj₀ is fixed. Then (3.1) becomes

b₁+b_j0x_j0+X

j6=j0

b_jx_{j v}

< H(¯x)^−ε. (3.5)

Since we are assuming that no proper subsum in (3.1) vanishes, b1+bj₀xj₀6=0 and no proper subsum in (3.5) vanishes either. Let ¯x⁰=(xj:j6=j0). ThenH(¯x⁰)6H(¯x). Hence, by the inductive assumption, the number of non-degenerate solutions ¯x⁰of (3.5) is finite, and this implies the proposition in this case.

Now suppose that c1=0 in (3.3). One ofc2, ..., csis non-zero, and for simplicity, we assume thatcs6=0. Then combining (3.1) with (3.3), we obtain that

b1+

s−1

X

j=2

(bj−cjbsc⁻¹_s )xj

_v

< H(¯x)^−ε. (3.6) Given a solution ¯xof (3.6), we pick a minimal J⊂{2, ..., s−1}such that

b1+X

j∈J

(bj−cjbsc⁻¹_s )xj

_v

< H(¯x)^−ε, (3.7) and no proper subsum ofb1+P

j∈J(bj−cjbsc⁻¹_s )xj vanishes. Let ¯x⁰=(xj:j∈J). Since H(¯x⁰)6H(¯x), it follows from the inductive hypothesis that ¯x⁰belongs to a fixed finite set.

This proves that for every solution ¯xof (3.1) there existsj0=2, ..., ssuch thatxj₀ belongs to a fixed finite set. Now we can finish the argument as in the previous paragraph, and this completes the proof of the proposition.

3.2. Higher-order mixing under irreducibility condition

We define the subspace W in L(G), the set of characters X0 and the eigenvectors wχ

withχ∈X0 as in§2.5.

In this section we assume that Dπ(W) is not contained in any proper rational subspace. Let{w1, ..., wk} be a fixed basis ofW. Consider a box map

ι:B−!L(G), t7−!

k

X

i=1

tiwi, whereB=[0, T1]×...×[0, Tk].

Lemma 3.2. Let f1, ..., fs∈C(X), u1, ..., us∈L(G), β1, ..., βs be automorphisms of Gsuch that βi=idof G/G⁰, z1, ... zs∈Z^l, v1, ..., vs∈L(G),and x1, ..., xs∈X. Then

1

|B|

Z

B

^s Y

i=1

fi(exp(ui)βi(α(zi)(exp(vi+ι(t))))xi)

dt=

s

Y

i=1

Z

X

fidµ+o(1) as min{kzik,kzi−zjk:i6=j}!∞. Moreover, the convergence is uniform over u₁, ..., u_s, β₁, ..., β_s, v₁, ..., v_s, x₁, ... x_s, and functions f₁, ..., f_s with kfik_Cθ1.