3 Exterior powers of the reflection representation

The Smallest Singular Values

and Vector-Valued Jack Polynomials

Charles F. DUNKL

Department of Mathematics, University of Virginia, PO Box 400137, Charlottesville VA 22904-4137, USA E-mail: cfd5z@virginia.edu

URL: http://people.virginia.edu/~cfd5z/

Received June 15, 2018, in final form October 22, 2018; Published online October 25, 2018 https://doi.org/10.3842/SIGMA.2018.115

Abstract. There is a space of vector-valued nonsymmetric Jack polynomials associated with any irreducible representation of a symmetric group. Singular polynomials for the smallest singular values are constructed in terms of the Jack polynomials. The smallest singular values bound the region of positivity of the bilinear symmetric form for which the Jack polynomials are mutually orthogonal. As background there are some results about general finite reflection groups and singular values in the context of standard modules of the rational Cherednik algebra.

Key words: nonsymmetric Jack polynomials; standard modules; Young tableaux 2010 Mathematics Subject Classification: 33C52; 20F55; 05E35; 05E10

1 Introduction

SupposeW is the finite reflection group generated by the reflections in the reduced root systemR.

This meansRis a finite set of nonzero vectors inR^N such that u, v∈R impliesRu∩R={±u}

and vσ_u ∈R whereσ_u is the reflection x7→x−2^hx,vi_hv,viv and h·,·iis the standard inner product.

This implies hxσ_v, yσvi=hx, yi for allx, y∈R^N and the groupW generated by{σ_v:v∈R} is a finite group of orthogonal transformations of R^N. For a fixed vector b0 such that hu, b₀i 6= 0 for all u ∈ R there is the decomposition R = R₊ ∪R− with R₊ := {u ∈ R: hu, b₀i > 0}.

The set R+ serves as index set for the reflections in W. In Section 3 it is assumed that span_R(R) = R^N, while the other sections concerning the symmetric group use the root sys- tem AN−1 whose span is

n

x∈R^N:

N

P

i=1

xi = 0 o

. The group W is represented on the spaceP of polynomials in x = (x1, . . . , x_N) by wp(x) =p(xw) for w∈ W. Denote N0 := {0,1,2, . . .} and forα∈N^N0 let |α|:=

N

P

i=1

αi and x^α :=

N

Q

i=1

x^α_iⁱ, a monomial. ThenP := span

x^α:α∈N^N0 and P_n:= span

x^α:α∈N^N0 ,|α|=n the space of polynomials homogeneous of degreen. Letκ be a parameter (called multiplicity function), a function onR constant onW-orbits. For indecom- posable groupsW there are at most two orbits in R(two for typesB_N,F₄ andI(2k), otherwise one for typeAN,DN,Em,I(2k+ 1)) then the Dunkl operators {D_i: 1≤i≤N}are defined by

D_ip(x) = ∂p(x)

∂xi

+ X

v∈R+

κ(v)p(x)−p(xσ_v) hx, vi v_i.

This paper is a contribution to the Special Issue on the Representation Theory of the Symmetric Groups and Related Topics. The full collection is available athttps://www.emis.de/journals/SIGMA/symmetric-groups- 2018.html

Then D_iD_j = D_jD_i for 1 ≤ i, j ≤ N, D_i maps P_n to P_n−1, and the Laplacian ∆κ :=

N

P

i=1

D²_i satisfies

∆κf(x) = ∆f(x) + X

v∈R+

κ(v)

2h∇f(x), vi

hx, vi − |v|²f(x)−f(xσv) hx, vi²

.

The abstract algebra generated by W, D_i, and multiplication by xi, 1 ≤ i ≤ N, acting on P is the rational Cherednik algebra. There are two W-invariant bilinear symmetric forms of interest here, denotedh·,·i_κ and h·,·i_κ,G. The first one (called thecontravariant form) satisfies hD_if, gi_κ =hf, x_igi_κ for all i and f, g ∈ P and hf, gi_κ = 0 if f, g are homogeneous of different degrees; also h1,1i_κ = 1,hwf, wgi_κ=hf, gi_κ forw∈W. The Gaussian form is derived from the first one byhf, gi_κ,G:=

e^∆κ/2f, e^∆κ/2g

κ. This form satisfieshD_if, gi_κ,G=hf,(x_i−D_i)gi_κ,Gfor alli, and thus multiplication byxi is self-adjoint becausehf, x_igi_κ,G=hD_if, gi_κ,G+hf,D_igi_κ,G. For certain constant values of κ the Gaussian form is realized as an integral with respect to a finite positive measure on R^N, in fact

hf, gi_κ,G=cκ

Z

R^N

f(x)g(x) Y

v∈R₊

|hx, vi|^2κ(v)e^−|x|2/2dmN(x),

wherem_N is Lebesgue measure onR^N (see [2, Theorem 3.10]). The constantc_κis a normalizing constant to matchh1,1i_κ,G= 1. The explanation of the value ofcκis in terms of thefundamental degrees ofW. By a theorem of Chevalley the ring ofW-invariant polynomials is generated byN algebraically independent homogeneous polynomials of degreesd₁ ≤d₂ ≤ · · · ≤d_N (generally ‘<’

holds), and these are the fundamental degrees (see [13, Section 3.5]). They satisfy

N

Q

i=1

d_i = #W and

N

P

i=1

(di−1) = #R+. The Macdonald–Mehta–Selberg integral formula is Z

R^N

Y

v∈R+

|hx, vi|^2κe^−|x|2/2dm_N(x) =c

N

Y

i=1

Γ(1 +d_iκ) Γ(1 +κ) ,

where c is independent of κ. There is a version of this for the B_N and F₄ types. Etingof [8, Theorem 3.1] gave a proof of the formula valid for all finite reflection groups. The integral shows that the measure is finite and positive for κ > −_d¹

N. Henceforth we consider only the one-parameter situation with W having just one conjugacy class of reflections.

This number−_d¹

N appears in another context. Suppose for some specific rational value of κ there exists a nonconstant polynomial p for which D_ip = 0 for 1 ≤ i ≤ N, then p is called a singular polynomial and κ is a singular value. We can assume that p is homogeneous. In this case hx^α, p(x)i_κ = D

1,

N

Q

i=1

D_i^αip(x)E

κ = 0 for all α ∈ N^N₀ with α 6= (0, . . . ,0) and thus hf, pi_κ = 0 for all f ∈ P. Furthermore ∆κp = 0 implying e^∆κ/2p = p and hp, pi_κ,G = 0. It follows that κ ≤ −_d¹

N (taking κ constant). In fact the smallest (in absolute value) singular value is indeed−_d¹

N [5, Theorem 4.9]. The theory can be extended to polynomials taking values in modules of W. Suppose τ is an irreducible orthogonal representation of W on a (finite- dimensional) real vector space V with basis {u_i: 1≤i≤dimV}. The space P_τ := P ⊗V has the basis

x^α⊗u_i:α∈N^N₀ ,1≤i≤dimV . There is a representation of W on P_τ defined to be the linear extension of

w7→w(p(x)⊗u) :=p(xw)⊗(τ(w)u), p∈ P, u∈V.

The associated Dunkl operators are the linear extensions of D_i(p(x)⊗u) = ∂p(x)

∂x_i ⊗u+κ X

v∈R₊

p(x)−p(xσ_v)

hx, vi v_i⊗(τ(σ_v)u).

The first bilinear form is a modification of the scalar one: let h·,·i_V be a W-invariant inner product on V, that is, hτ(w)u1, τ(w)u2i_V = hu₁, u2i_V for all u1, u2 ∈ V, w ∈ W; this form is unique up to multiplication by a constant. The symmetric form h·,·i_κ satisfies (i) h1⊗u₁,1⊗ u₂i_κ = hu₁, u₂i_V for u₁, u₂ ∈ V, (ii) hwf, wgi_κ = hf, gi_κ for f, g ∈ P_τ and w ∈ W, (iii) if f, g∈ P_τ are homogeneous of different degrees then hf, gi_κ = 0, (iv)hD_if, gi_κ=hf, x_igi_κ for all i. As a consequence suppose α ∈N^N₀ ,|α|=n,u∈ V and f ∈ P_τ is homogeneous of degreen, then f₀ :=

N

Q

i=1

D_i^αif(x) ∈ V and hx^α ⊗u, fi_κ = hu, f₀i_V. The Gaussian form is defined by hf, gi_κ,G:=

e^∆κ/2f, e^∆κ/2g

κ as in the scalar case, and the definition of singular polynomials is the same (D_if = 0 for alli, some specific value ofκ). The interesting question is for whatκis the form h·,·i_κ positive-definite; this property is equivalent to positivity of the Gaussian form. The property hf, x_igi_κ,G=hx_if, gi_κ,Gsuggests that this form can be realized as an integral over R^N with a positive matrix-valued measure. Shelley-Abrahamson [16] proved there is a small interval forκabout zero for which this occurs. The interval is a subset of the interval for which the form is positive. The containment may be proper but the question of equality is not settled as yet.

It is the purpose of this note to show that the positivity interval is bounded by the smallest singular values, to illustrate the theory by constructing singular polynomials for exterior powers of the reflection representation of anyW, and to construct vector-valued Jack polynomials which specialize to singular polynomials for the symmetric groups. In this situation the representation is determined by a partition τ of N and the smallest singular values are ±_h¹

τ where h_τ is the longest hook-length of the Ferrers diagram of τ (see Etingof and Stoica [9, Section 5]). The isotype (that is, a partition of N) of these singular polynomials is determined.

There are two ways of finding singular polynomials, either define them directly (as in [9]) or describe the nonsymmetric Jack polynomials which become singular when specialized to the appropriate parameter value. Feigin and Silantyev [10] found explicit formulas for all singular polynomials which span a W-module isomorphic to the reflection representation of W.

The presentation starts with the result on the positivity of the Gaussian form, then the defini- tion and properties ofP_τ, the exterior powers of the reflection representation, the nonsymmetric Jack polynomials, results about the action of D_i and the construction of the singular polyno- mials. The theory of vector-valued nonsymmetric Jack polynomials, originated by Griffeth [11], allows detailed analyses ofP_τ. In fact he constructed these polynomials for any groupG(r,1, N), the group of N ×N monomial matrices whose nonzero entries are r^th roots of unity. In [12, Section 5] he determined the unitarity locus associated to the contravariant forms associated to these polynomials. These are regions in the parameter space R^r and the highest-dimensional components can be shown to be the regions of positivity.

2 Region of positivity of the Gaussian form

Fix an irreducible representation τ of W. The formh·,·i_κ is normalized by h1⊗u1,1⊗u2i_κ = hu₁, u₂i_V where h·,·i_V is a W-invariant bilinear positive symmetric form on V (it is unique up to a multiplicative constant).

Definition 2.1. Let Ω denote the region ofκ∈Rfor which hf, fi_κ ≥0 for all f ∈ P_τ. The following is due to Shelley-Abrahamson [16].

Theorem 2.2. The region Ω contains a neighborhood ofκ= 0.

This result includes the existence of a matrix measure on R^N which realizes the Gaussian form.

Lemma 2.3. Suppose for some κ ∈ Ω there is a polynomial f ∈ P_τ such that f 6= 0 and hf, fi_κ = 0 then the space X_f := span{wf:w∈W} can be decomposed as a sum of irreducible W-modules and g∈X_f implies hg, pi_κ= 0 for all p∈ P_τ.

Proof . The decomposability is a group-theoretic property. By the W-invariance property of h·,·i_κ it follows thathwf, wfi_κ = 0 for allw. The Cauchy–Schwartz inequality forh·,·i_κ is valid because κ∈Ω thus |hwf, pi_κ|² ≤ hwf, wfi_κhp, pi_κ = 0 for all w∈W and p∈ P_τ. In particular hw₁f +w₂f, pi_κ= 0 for any w₁, w₂ and thusg∈X_f implies hg, pi_κ = 0.

We will show that the set of singular values is a subset of a set of rational numbers with no accumulation point, that is, there is a minimum nonzero distance between elements.

Proposition 2.4. The eigenvalues of the class P

v∈R+

σv considered as a(central) transformation of the group algebra RW are integers in the interval [−#R₊,#R+].

Proof . The basic idea is that the solutions of the characteristic equation are algebraic integers.

The details are in [7, p. 194].

Because the right regular representation of W on RW is a direct sum of all irreducible representations ofW the integer property of eigenvalues applies to P

v∈R+

ρ(σv) for any irreducible representation ρ. Since ρ is irreducible and P

v∈R₊

ρ(σv) is central there is just one eigenvalue, denoted byε(ρ). Denote the set of equivalence classes of irreducible representations ofW byWc. Proposition 2.5. Suppose f ∈ P_τ then

PN i=1

x_iD_if = PN i=1

x_i∂x^∂f

i +κ

ε(τ)f − P

v∈R₊

σ_vf

. If f is singular and homogeneous of degree nthen κ= _{ε(ρ)−ε(τ)}ⁿ where ρ∈Wc.

Proof . Letp∈ P_n and u∈V then

N

X

i=1

x_iD_i(p(x)⊗u) =

N

X

i=1

x_i∂p(x)

∂xi

⊗u+κ X

v∈R+

(p(x)−p(xσ_v))⊗(τ(σ_v)u)

=np(x)⊗u+κ

ε(τ)p(x)⊗u− X

v∈R+

σ_v(p(x)⊗u)

.

The statement P

v∈R+

p(x)⊗(τ(σ_v)u) =ε(τ)p(x)⊗ufollows from the fact thatV is the represen- tation space forτ. The relation is extended to all ofP_τ by linearity. That is, iff ∈ P_n⊗V then

N

P

i=1

xiD_i(f(x)) =nf(x) +κε(τ)f(x)−κ P

v∈R+

σvf(x). If f is singular then it must be an eigen- function of P

v∈R₊

σ_v. The space span{wf: w ∈ W} consists of singular polynomials (same κ) and can be decomposed into irreducibleW-submodules, thus the eigenvalues of P

v∈R+

σ_v are ele- ments of the set

ε(ρ) :ρ ∈ cW . Hence there is some ρ ∈Wc such that P

v∈R+

σvf = ε(ρ)f and 0 =nf+κ(ε(τ)−ε(ρ))f. (The caseε(τ) =ε(ρ) is impossible.)

It is possible thatε(ρ1) =ε(ρ2) for someρ1, ρ2 ∈Wc withρ1 6=ρ2, but thenf can be decom- posed into two components, each being singular (by convolution with the respective characters).

The minimum distance between two singular values is bounded below by max

ρ |ε(τ)−ε(ρ)|−1

. Recall {ε(ρ)}is a set of integers contained in [−#R₊,#R₊].

For each n≥1 restrict the form h·,·i_κ toP_n⊗V. The condition that the form is positive- definite is that the leading principal minors of the Gram matrix are positive (for example use the basis

x^α⊗u_i:|α|=n, 1≤i≤dimV ). The minors are polynomials inκand are positive in a neighborhood of 0. Let zn denote the positive zero of any of the minors closest to 0, that is the form is positive for 0 ≤ κ < z_n and is positive-semidefinite for κ = z_n and there exists f_n∈ P_n⊗V such thatf_n6= 0 and hf_n, f_ni_κ = 0 (which implieshf_n, gi_κ = 0 for any g∈ P_n⊗V by the Cauchy–Schwartz inequality). If there are no positive zeros set zn=∞.

(The reason for the following careful argument is to avoid the hypothetical situation z_n = 1 +_n¹, minzn= 1 and there is no nonzero polynomial f withhf, fi_κ= 0 for κ= 1.)

Lemma 2.6. Suppose z_n> z_n+1 thenz_n+1 is a singular value.

Proof . Setκ=z_n+1. By the definition ofz_n+1 and properties of the form 0 =

fn+1,

N

X

i=1

xiD_ifn+1

κ

=

N

X

i=1

hD_ifn+1,D_ifn+1i_κ.

By hypothesis the form is positive-definite on P_n⊗V for κ = z_n+1 < z_n. Thus f_n+1 is sin-

gular.

Define the subsequence {z_ni} by n₁ = min{n:z_n < ∞} and n_i+1 = min{n: n > n_i, z_n <

zn−1} (essentially the points of decrease of the sequence). If there are no positive eigenvalues then eachzn=∞and the form is positive-definite for κ≥0. Now assume there is at least one z_n<∞. Eachz_n≥z_ni for somei.

Theorem 2.7. Let z0 = min{z_ni:i≥1} then z0 =znj for some j, the form h·,·i_κ is positive- definite for 0≤κ < z₀ and z₀ is a singular value.

Proof . By the lemma the subsequence consists of singular values. The spacing of singular values implies there is no accumulation point thus the minimum z₀ is achieved at one of the values znj. Hence there existsf ∈ P_nj ⊗V such thatf 6= 0 and f is singular for κ=z0. The same argument can be applied to negativeκ: let z_n⁰ be the negative zero closest to 0 of the leading principal minors of the form restricted to P_n⊗V so the form is positive-definite for z_n⁰ < κ≤ 0; ifz_n+1⁰ > z_n⁰ then z_n+1⁰ is a singular value and so is z⁰₀ = max{z_n⁰} (excluding the situation of no negative singular values wherez₀⁰ =−∞).

To summarize there is an intervalz⁰₀< κ < z0 for which h·,·i_κ is positive-definite and z0,z₀⁰ are singular values if finite, respectively.

3 Exterior powers of the reflection representation

Suppose W has only one conjugacy class of reflections and span_R(R) = R^N. Specialize τ to the reflection representation of W on V =R^N. (The previous two statements imply that W is indecomposable and the reflection representation is irreducible.) Let ∧^m(V) =V ∧V ∧ · · · ∧V (m factors) with 1 ≤ m ≤ N. We will show that P₁ ⊗ ∧^m(V) has singular polynomials for κ = ±_d¹

N, where dN is the largest fundamental degree of W (also see [9, Corollary 4.2]), and 1 ≤m < N. Ciubotaru [1, Section 5] proved a necessary condition for the region of positivity

for any W (in fact, also for complex reflection groups) and any irreducible W-moduleU, which involves the decomposition ofU ⊗ ∧^m(V) intoW-irreducible subspaces.

Let{u_i: 1≤i≤N}be the standard orthonormal basis of V, and letx=

N

P

i=1

x_i⊗u_i. Lemma 3.1. Fora, b∈V the symmetric bilinear form P

v∈R₊

ha,vihb,vi

|v|² = ^#R_N⁺ha, bi. In particular P

v∈R+

vivj

|v|² = ^#R_N⁺δ_ij.

Proof . For a, b∈V define [a, b] := P

v∈R+

1

[v]²ha, vihb, vi then [aσ_u, bσ_u] = X

v∈R₊

1

|v|²haσ_u, vihbσ_u, vi= X

v∈R₊

1

|v|²ha, vσ_uihb, vσ_ui= [a, b]

for all u ∈ R₊. Thus [aw, bw] = [a, b] for all w ∈W and the matrix representing this bilinear form with respect to the basis {u_i} commutes with each w. By hypothesis τ is irreducible and by Schur’s lemma the matrix is a scalar multiple of the identity and [a, b] =cha, bi for all a, b ∈ V. The matrix representing the form ^ha,vihb,vi_|v|2 has trace 1 thus the trace for the sum overv ∈R₊ is #R₊. The formha, bicorresponds to the identity matrix and has traceN. The

other conclusion follows from vivj =hu_i, vihu_j, vi.

Henceforth assume|v|² = 2 for allv∈R, and setγ := 2^#R_N⁺, called theCoxeter number (see [13, Section 3.18]); thus P

v∈R+

v_iv_j =γδ_ij. The computations use a boundary operator.

Definition 3.2. Supposea, b1, b2, . . . , bm∈V then

∂(a)(b1∧b2∧ · · · ∧bm) :=

m

X

i=1

(−1)ⁱ⁻¹ha, b_iib₁∧b2∧ · · · ∧bbi∧ · · · ∧bm,

where the caret indicates the omitted factor. The operator ∂(a) is extended to all of∧^m(V) by linearity.

It can be checked that∂(a) is well-defined, for example suppose thatbm =

m−1

P

i=1

cibi then

∂(a)(b1∧b2∧ · · · ∧bm) = (−1)^m−1 ha, b_mi −

m−1

X

i=1

ciha, b_ii

!

b1∧ · · · ∧bm−1 = 0.

Lemma 3.3. Suppose a, b₀ ∈ V and b ∈ ∧^m(V) then ∂(a)(b₀∧b) = ha, b₀ib−b₀∧∂(a)b and

∂(a)²(b₀∧b) = 0.

Proof . The first part follows directly from the definition. Apply ∂(a) to both sides of the equation

∂(a)²(b0∧b) =ha, b₀i∂(a)b−

ha, b₀i∂(a)b−b0∧∂(a)²b =b0∧∂(a)²b.

Setb=b₁∧ · · · ∧b_m and repeatedly use this relation to show∂(a)²b=b₁∧ · · · ∧∂(a)²b_m= 0.

Denote the exterior power ofτ on ∧^m(V) byτ_m. The operator P

v∈R+

τ_m(σ_v) acts as multipli- cation by ε(τ_m) = ^N₂ −m

γ on ∧^m(V). (Assume thatv =√

2u₁ ∈R and consider the action of τ_m(σ_v) on the basis

{u_i1 ∧ui2 ∧ · · · ∧uim: 1≤i1< i2 <· · ·< im≤N};

there are ^N_m−1⁻¹

eigenvectors for the eigenvalue−1 and ^N_m⁻¹

eigenvectors for 1 thus Tr(τm(σv))

= ^N−1_m

− ^N−1_m−1

, thenε(τm) = #R+Tr(τm(σv))/dim∧^m(V) = ¹₂γ(N −2m).) Proposition 3.4. Suppose v∈R and b∈ ∧^m(V) then τ_m(σ_v)b=b−v∧∂(v)b.

Proof . Letb=b1∧b2∧ · · · ∧bm. By definition

τm(σv)b= (b1σv)∧ · · · ∧(bmσv) = (b1− hb₁, viv)∧ · · · ∧(bm− hb_m, viv)

=b− hb₁, viv∧b2∧ · · · ∧bm− hb₂, vib₁∧v∧ · · · ∧bm− · · ·

=b−v∧(hb₁, vib₂∧ · · · ∧bm− hb₂, vib₁∧b3∧ · · ·) =b−v∧∂(v)b.

To compute the terms inD_ip we find ^xi−(xσ_hx,vi^v)i =v_i and ha,xi−ha,xσ_vi

hx,vi =ha, vi. Note x and x are different objects, with different transformation rules forW, in factσ_vx=x, because

σ_vx=

N

X

i=1

(x_i− hx, viv_i)⊗(u_i−v_iv) =x−2hx, viv+hx, vi|v|²v=x.

Theorem 3.5. Suppose a ∈ V, b ∈ ∧^m(V) then x∧b ∈ P₁ ⊗ ∧^m+1(V), PN i=1

a_iD_i(x∧b) = (1−γκ)a∧b, andx∧bis singular for κ= 1/γ.

Proof . Supposea∈V then

N

X

i=1

aiD_i(x∧b) =

N

X

i=1

aiui∧b+κ X

v∈R+

N

X

j=1

ha, viv_jτm(σv)(uj∧b)

=a∧b+κ X

v∈R₊

ha, viτ_m(σ_v)(v∧b)

=a∧b+κ X

v∈R+

ha, vi(v∧b−v∧∂(v)(v∧b))

=a∧b+κ X

v∈R₊

ha, vi(v∧b− hv, viv∧b+v∧v∧∂(v))

=a∧b−κ X

v∈R+

ha, viv∧b= (1−γκ)a∧b,

because hv, vi= 2 and P

v∈R+

ha, viv=

N

P

i,j=1

P

v∈R+

aivivjuj =γ

N

P

i=1

aiui =γa.

Theorem 3.6. Suppose a∈V, b∈ ∧^m(V) then

∂(x)b∈ P₁⊗ ∧^m−1(V),

N

X

i=1

aiD_i∂(x)b= (1 +γκ)∂(a)b, and ∂(x)b is singular for κ=−1/γ.

Proof . Assume b=b₁∧ · · · ∧b_m withb₁, . . . , b_m ∈V. Then

N

X

i=1

aiD_i∂(x)b=

N

X

i=1

ai

∂

∂x_i

m

X

j=1

(−1)^j−1hx, b_ji ⊗ b1∧ · · · ∧bbj∧ · · ·

+κ X

v∈R+

ha, vi

m

X

j=1

(−1)^j−1hv, b_jiτ_m(σv) b1∧ · · · ∧bbj∧ · · ·

=

m

X

j=1

(−1)^j−1ha, b_ji ⊗ b1∧ · · · ∧bbj ∧ · · ·

+κ X

v∈R+

ha, viτ_m(σv)(∂(v)b)

=∂(a)b+κ X

v∈R₊

ha, vi ∂(v)b−v∧∂(v)²b

= (1 +γκ)∂(a)b,

because ∂(v)²b= 0 (Lemma 3.3) and X

v∈R+

ha, vi∂(v)b=

m

X

j=1

(−1)^j−1 X

v∈R+

ha, vihb_j, vi b1∧ · · · ∧bbj∧ · · ·

=γ∂(a)b.

Here is a table with data on the indecomposable groups with one conjugacy class of reflections.

The subscripts indicate the rankN of the group

W I2(2k+ 1) AN H3 H4 E6 E7 E8

#R+ 2k+ 1 ^N(N+1)₂ 15 60 36 63 120

d_N 2k+ 1 N + 1 10 30 12 18 30

.

Thus γ = ^2#R_N⁺ =dN, the largest fundamental degree, also called the Coxeter number (see [13, Section 3.18]).

Considering the known situation for the symmetric groups and for the reflection representa- tion it appears that for a large collection of representationsτ of degree greater than one that the interval z₀⁰ < κ < z0 of positivity is symmetric, z₀⁰ =−z₀, andz0 ≥ _d¹

N. However this is not al- ways the case: there is a nonsymmetric positivity interval arising in two degree 3 representations of the icosahedral group H3 (see [4]).

4 Representations of the symmetric groups

The symmetric group S_N, the set of permutations of{1,2, . . . , N}, acts onC^N by permutation of coordinates. The space of polynomials P := span_R(κ)

x^α:α∈N^N0 whereκ is a parameter.

The action ofS_N is extended to polynomials bywp(x) =p(xw) where (xw)_i =x_w(i) (considerx as a row vector and w as a permutation matrix, [w]_ij = δ_i,w(j), then xw = x[w]). This is a representation of S_N, that is, w1(w2p)(x) = (w2p)(xw1) = p(xw1w2) = (w1w2)p(x) for all w₁, w₂ ∈ S_N.

Furthermore S_N is generated by reflections in the mirrors {x: x_i = x_j} for 1≤i < j ≤N. These are transpositions,denoted by (i, j), so that x(i, j) denotes the result of interchangingxi

and x_j. Define the S_N-action onα∈Z^N so that (xw)^α=x^wα (xw)^α =

N

Y

i=1

x^α_w(i)ⁱ =

N

Y

j=1

x^α_j^w−1(j),

that is (wα)_i=α_w⁻¹_(i).

Thesimple reflections si:= (i, i+ 1), 1≤i≤N−1, generate S_N. They are the key devices for applying inductive methods, and satisfy the braid relations:

sisj =sjsi, |i−j| ≥2, sisi+1si=si+1sisi+1.

We consider the situation where the group S_N acts on the range as well as on the domain of the polynomials. We use vector spaces, called S_N-modules, on which S_N has an irreducible orthogonal representation τ: S_N → Om(R) (τ(w)⁻¹ = τ w⁻¹

= τ(w)^T). See James and Kerber [14] for representation theory, including a modern discussion of Young’s methods.

Denote the set ofpartitions N^N,+₀ :=

λ∈N^N0 :λ1≥λ2 ≥ · · · ≥λ_N .

We identify τ with a partition of N given the same label, that is τ ∈N^N,+₀ and |τ|=N. The length of τ is `(τ) := max{i: τi > 0}. There is a Ferrers diagram of shape τ (also given the same label), with boxes at points (i, j) with 1≤i≤`(τ) and 1 ≤j ≤τ_i. Atableau of shape τ is a filling of the boxes with numbers, and areverse standard Young tableau (RSYT) is a filling with the numbers {1,2, . . . , N} so that the entries decrease in each row and each column.

Definition 4.1. The hook-length of the node (i, j)∈τ is defined to be h(i, j) :=τi−j+ #{k:i < k≤`(τ), j≤τ_k}+ 1,

and the maximum hook-length is hτ :=h(1,1) =τ1+`(τ)−1.

Denote the set of RSYT’s of shapeτ by Y(τ) and let V_τ = span_R(κ){T:T ∈ Y(τ)}

with orthogonal basis Y(τ). For 1 ≤ i ≤ N and T ∈ Y(τ) the entry i is at coordinates (row(i, T),col(i, T)) and thecontent isc(i, T) := col(i, T)−row(i, T). EachT ∈ Y(τ) is uniquely determined by itscontent vector [c(i, T)]^N_i=1. There is an irreducible representation ofS_N onVτ

also denoted byτ (slight abuse of notation). To specify the action ofτ it suffices for our purposes to give only the formulae forτ(s_i):

1) row(i, T) = row(i+ 1, T) (implying col(i, T) = col(i+ 1, T) + 1 andc(i, T)−c(i+ 1, T) = 1) then

τ(si)T =T;

2) col(i, T) = col(i+1, T) (implying row(i, T) = row(i+1, T)+1 andc(i, T)−c(i+1, T) =−1) then

τ(si)T =−T;

3) row(i, T)<row(i+ 1, T) and col(i, T)>col(i+ 1, T). In this case

c(i, T)−c(i+ 1, T) = (col(i, T)−col(i+ 1, T)) + (row(i+ 1, T)−row(i, T))≥2, and T⁽ⁱ⁾, denoting the tableau obtained from T by exchanging i and i+ 1, is an element of Y(τ) and

τ(si)T =T⁽ⁱ⁾+ 1

c(i, T)−c(i+ 1, T)T,

4) c(i, T)−c(i+ 1, T)≤ −2, thus row(i, T)>row(i+ 1, T) and col(i, T)<col(i+ 1, T) then withb=c(i, T)−c(i+ 1, T),

τ(si)T =

1− 1 b²

T⁽ⁱ⁾+1 bT.

The formulas in (4) are consequences of those in (3) by interchangingT andT⁽ⁱ⁾and applying the relations τ(s_i)² =I (where I denotes the identity operator onV_τ). TheS_N-invariant inner product onV_τ is defined by

hT, T⁰i₀ :=δ_{T ,T}⁰× Y

1≤i<j≤N, c(i,T)≤c(j,T)−2

1− 1

(c(i, T)−c(j, T))²

, T, T⁰ ∈ Y(τ).

It is unique up to multiplication by a constant.

The Jucys–Murphy elements ωi :=

N

P

j=i+1

(i, j) satisfy

N

P

j=i+1

τ((i, j))T = c(i, T)T. Thus P

1≤i<j≤N

τ((i, j)) acts onV_τ as multiplication by

ε(τ) =

N

X

j=1

c(j, T) = 1 2

`(τ)

X

i=1

τi(τi−2i+ 1) (independent of T ∈ Y(τ)).

5 Vector-valued Jack polynomials

For a given partitionτ ofNthere is a space of vector-valued nonsymmetric Jack polynomials, also called a standard module of the rational Cherednik algebra. The nonsymmetric vector-valued Jack polynomials (NSJP) form a basis ofP_τ =P ⊗ V_τ, the space ofV_τ valued polynomials inx, equipped with the S_N action

w x^α⊗T

:= (xw)^α⊗τ(w)T, α∈N^N0 , T ∈ Y(τ), which is extended by linearity to

wp(x) =τ(w)p(xw), p∈ P_τ.

Definition 5.1. The Dunkl and Cherednik–Dunkl operators are (1≤i≤N, p∈ P, T ∈ Y(τ)) D_i(p(x)⊗T) := ∂p(x)

∂xi

⊗T+κX

j6=i

p(x)−p(x(i, j)) xi−xj

⊗τ((i, j))T,

U_i(p(x)⊗T) :=D_i(x_ip(x)⊗T)−κ

i−1

X

j=1

p(x(i, j))⊗τ((i, j))T, extended by linearity to all ofP_τ.

The commutation relations analogous to the scalar case hold, that is, D_iD_j =D_jD_i, U_iU_j =U_jU_i, 1≤i, j≤N,

wD_i =D_w(i)w, ∀w∈ S_N, s_jU_i =U_is_j, j6=i−1, i, siU_isi =U_i+1+κsi, U_isi =siU_i+1+κ, U_i+1si =siU_i−κ.

The simultaneous eigenfunctions of {U_i}are called (vector-valued) nonsymmetric Jack polyno- mials (NSJP). For genericκthese eigenfunctions form a basis of P_τ (generic means thatκ6= ^m_n

wherem, n∈Zand 1≤n≤N). They have a triangularity property with respect to the partial order on compositions, which is derived from the dominance order:

α≺β⇐⇒

i

X

j=1

αj ≤

i

X

j=1

βj, 1≤i≤N, α6=β, αβ⇐⇒(|α|=|β|)∧

α⁺≺β⁺

∨ α⁺=β⁺∧α≺β .

There is a subtlety in the leading terms, which relies on the rank function:

Definition 5.2. Forα∈N^N₀ , 1≤i≤N

r_α(i) = #{j:α_j > α_i}+ #{j: 1≤j≤i, α_j =α_i}, then rα∈ S_N.

A consequence is that r_αα = α⁺, the nonincreasing rearrangement of α, for any α ∈ N^N0 . For example if α = (1,2,1,5,4) then r_α = [4,3,5,1,2] and r_αα = α⁺ = (5,4,2,1,1) (recall wαi =α_w⁻¹_(i)). Also rα=I if and only ifα is a partition (α1 ≥α2 ≥ · · · ≥αN).

For eachα∈N^N0 andT ∈ Y(τ) there is a NSJPζα,T with leading termx^α⊗τ r_α⁻¹

T, that is, ζ_α,T =x^α⊗τ r⁻¹_α

T+X

αβ

x^β⊗t_αβ(κ), t_αβ(κ)∈V_τ, U_iζ_α,T = (α_i+ 1 +κc(r_α(i), T))ζ_α,T, 1≤i≤N.

The list of eigenvalues is called the spectral vector ξα,T := [αi+ 1 +κc(rα(i), T)]^N_i=1.

The NSJP’s can be constructed by means of a Yang–Baxter graph. The details are in [6];

this paper has several figures illustrating some typical graphs.

A node consists of (α, T, ξα.T, rα, ζα,T),

where α∈N^N₀ ,T ∈ Y(τ), ξ_α,T is the spectral vector. The root is 0, T₀,[1 +κc(i, T₀)]^N_i=1, I,1⊗T₀

,

whereT0 is formed by enteringN, N−1, . . . ,1 column-by-column in the Ferrers diagram. Proofs by induction in this context typically rely on sequences of applications of{τ(si)}and the inver- sion number: for T ∈ Y(τ) set

inv(T) := #{(i, j) :i < j, c(i, T)−c(j, T)≥2}.

If for particularT andithe relationc(i, T)−c(i+1, T)≥2 holds thenT⁽ⁱ⁾, the tableau formed by interchangingiandi+1 inTis also a RSYT (the relation is equivalent to row(i, T)<row(i+1, T) and col(i, T)>col(i+ 1, T); the RSYT property implies that these two inequalities are logically equivalent). In this case inv T⁽ⁱ⁾

= inv(T)−1. The inv-maximal tableau is T0 and the inv-minimal tableau is T₁ formed by enteringN, N −1, . . . ,1 row-by-row.

Steps in the YB-graph correspond toτ(si). There are several cases; we start with the situation αi=αi+1. These formulae restricted to 1⊗T (soα=0) are equivalent to the definition of the representationτ onV_τ. Throughout the hypotheses areα∈N^N₀ ,T ∈ Y(τ), 1≤i < N.

Case 1. αi =αi+1; definej:=rα(i) implying rα(i+ 1) =j+ 1 1) row(j, T) = row(j+ 1, T) thensiζα,T =ζα,T;

2) col(j, T) = col(j+ 1, T) thensiζα,T =−ζ_α,T;

3) row(j, T)<row(j+ 1, T) (thusc(j, T)−c(j+ 1, T)≥2) set

b⁰ := 1

c(j, T)−c(j+ 1, T) = κ

ξ_α,T(i)−ξ_α,T(i+ 1), then there is astep

(α, T, ξα,T, rα, ζα,T)−→^si α, T^(j), siξα,T, rα, ζ_α,T(j)

, ζ_α,T(j) =siζα,T −b⁰ζα,T, Note 0< b⁰ ≤ ¹₂ and τ(sj)T =T^(j)+b⁰T; furthermore the leading term is transformedsi x^α⊗ τ r_α⁻¹

T

= (xs_i)^α⊗τ s_ir⁻¹_α

T =x^α⊗τ r⁻¹_α

τ(s_j)T because s_ir⁻¹_α = r_α⁻¹s_j. The reciprocal relation is

s_iζ_α,T(j) =−b⁰ζ_α,T(j) + 1−b⁰² ζ_α,T. Case 2. αi+1 > αi, then with

b:= κ

ξα,T(i)−ξα,T(i+ 1) there is astep

(α, T, ξ_α,T, r_α, ζ_α,T)−→^si (s_iα, T, s_iξ_α,T, r_αs_i, ζ_siα,T), ζ_siα,T =s_iζ_α,T −bζ_α,T, and the reciprocal relation is

siζsiα,T =−bζ_siα,T + 1−b² ζα,T.

The reciprocal relations are derived from s²_i = 1. With the aim of letting κ take on certain rational values we examine the possible poles in the step rules arising from the factors

ξ_α,T(i)−ξ_α,T(i+ 1) =α_i−α_i+1+κ(c(r_α(i), T)−c(r_α(i+ 1), T)).

The extreme values of c(·, T) areτ₁−1 and 1−`(τ), and thus

|c(r_α(i), T)−c(rα(i+ 1), T)| ≤hτ−1.

Hence −_h¹

τ−1 < κ < _h¹

τ−1 and α_i 6=α_i+1 imply ξ_α,T(i)−ξ_α,T(i+ 1)6= 0 (in case α_i =α_i+1 the bound 0< b⁰≤ ¹₂ applies).

The other links in the YB-graph are degree-raising (affine) operations. Define

Φ(a₁, a₂, . . . , a_N) := (a₂, a₃, . . . , a_N, a₁+ 1), θ_m :=s₁s₂· · ·sm−1, 2≤m≤N, so that θm is the cyclic permutation (12. . . m). The cycle θN interacts with Φ and the rank function by r_Φα =r_αθ_N (that is, r_αθ_N(i) = r_α(i+ 1) =r_Φα(i) for 1≤i < N, and r_αθ_N(N) = rα(1) =rΦα(N)). Thejump is given by

(α, T, ξα,T, rα, ζα,T)−→^Φ Φα, T,Φξα,T, rαθN, xNθ_N⁻¹ζα,T

, ζΦα,T =xNθ⁻¹_N ζα,T. The leading term is x^Φα⊗τ θ_N⁻¹r_α⁻¹

T and θ⁻¹_N r⁻¹_α = (r_αθ_N)⁻¹. For example: α = (0,2,5,0), r_α= [3,2,1,4], Φα= (2,5,0,1),r_Φα= [2,1,4,3].

For any κ there is a unique bilinear symmetric S_N-invariant form on P_τ which satisfies (f, g∈ P_τ):

h1⊗T,1⊗T⁰i_κ =hT, T⁰i₀, T, T⁰ ∈ Y(τ),

hwf, wgi_κ=hf, gi_κ, w∈ S_N, hD_if, gi_κ =hf, x_igi_κ, 1≤i≤N.

As a consequence hx_if, x_igi_κ = hD_ix_if, gi_κ = hf,D_ix_igi_κ and U_i = D_ix_i −κ

i−1

P

j=1

(i, j) is self- adjoint; furthermore hζ_α,T, ζ_β,T⁰i_κ = 0 whenever (α, T) 6= (β, T⁰) because ξ_α,T 6= ξ_β,T⁰ for genericκ. The form is defined in terms ofhζ_α,T, ζα,Ti_κ and is extended by linearity and orthog- onality to all polynomials. It is a special case of a result of Griffeth [11]. The first ingredient is the formula for ζ_λ,T forλ∈N^N,+₀ (the Pochhammer symbol is (t)n=

n

Q

i=1

(t+i−1))

hζ_λ,T, ζ_λ,Ti_κ =hT, Ti₀

N

Y

i=1

(1 +κc(i, T))_λi

× Y

1≤i<j≤N λi−λj

Y

`=1

1−

κ

`+κ(c(i, T)−c(j, T)) 2!

. (5.1)

The second ingredient expresses the relationship between hζ_α,T, ζ_α,Ti_κ and hζ_α+,T, ζ_α+,Ti_κ. Let E(α, T) := Y

1≤i<j≤N αi<αj

1−

κ

αj−αi+κ(c(rα(j), T)−c(rα(i), T)) 2!

.

Then

hζ_α,T, ζα,Ti_κ=E(α, T)⁻¹hζ_α+,T, ζ_α⁺_,Ti_κ, α∈N^N₀ , T ∈ Y(τ). (5.2) From the bounds onc(i, T)−c(j, T) and the formulae it follows thathζ_α,T, ζ_α,Ti_κ>0 provided

−_h¹

τ < κ < _h¹

τ. Denote hf, fi_κ by kfk² for any generic value of κ (slight abuse of notation).

6 Differentiation formulae

First we prove formulae for D_jζ_α,T for `(α) ≤ j ≤ N (recall the length of α is `(α) :=

max{i: α_i > 0}). We need the commutation relations (part of the defining relations of the rational Cherednik algebra), p∈ P_τ:

D_i(xip)−xiD_ip=p+κ

N

X

j=1, j6=i

(i, j)p, (6.1)

D_i(xjp)−xjD_ip=−κ(i, j)p, i6=j. (6.2) Recall the Jucys–Murphy elements ωi :=

N

P

j=i+1

(i, j) for 1≤i < N andωN := 0.

Proposition 6.1. Suppose p∈ P_τ and 1≤i≤N then D_ip= 0 if and only if U_ip=p+κωip.

Proof . By the above U_ip=D_i(x_ip)−κX

j<i

(i, j)p=x_iD_ip+p+κ

N

X

j=1,j6=i

(i, j)p−κX

j<i

(i, j)p

=xiD_ip+p+κ

N

X

j=i+1

(i, j)p.

Corollary 6.2. Suppose α∈N^N0 , T ∈ Y(τ) and `(α)< N thenD<sub>i</sub>ζα,T = 0 for `(α)< i≤N.

Proof . By hypothesis rα(N) = N and ξα(N) = 1 +κc(N, T) = 1; also (1 +κωN)ζα,T = ζα,T

thus U_Nζ_α,T = ζ_α,Tp = ζ_α,T +κω_Nζ_α,T and D_Nζ_α,T = 0. Proceeding by induction suppose D_iζ_β,T⁰ = 0 for m ≤ i ≤ N with m > `(α) + 1 and all T<sup>0</sup> ∈ Y(τ) and β with `(β) ≤ `(α).

Then D_m−1ζ_β,T⁰ =sm−1D_msm−1ζ_β,T⁰. By the transformations in Case1(note βm−1 =βm = 0) sm−1ζ_β,T⁰ =±ζ_β,T⁰ or sm−1ζ_β,T⁰ is a linear combination (independent of κ) of ζ_β,T⁰ and ζ_β,T⁰⁰ whereT⁰⁰is the result of interchangingmandm−1 inT⁰. In any of these casesD_msm−1ζ_β,T⁰ = 0

by the induction hypothesis.

Suppose`(α) =m < N. We turn to the evaluation of D_mζ_α,T. Define αb= (αm−1, α1, α2, . . . , αm−1,0, . . . ,0).

The use of αbappeared in Knop [15] in a creation formula for nonsymmetric Macdonald polyno- mials. We prove the following in several steps:

Theorem 6.3. Suppose α∈N^N0 , T ∈ Y(τ) and `(α) =m < N then D_mζα,T = kζ_α,Tk²

kζ

α,Tb k²θ⁻¹_m ζ_α,Tb . Proposition 6.4. Suppose i6=j then

U_iD_j− D_jU_i =κD_min(i,j)(i, j).

Proof . Supposei < j then (by use of (6.2)) U_iD_j =D_ix_iD_j−κX

s<i

(i, s)D_j =D_i(D_jx_i+κ(i, j))−κD_jX

s<i

(i, s)

=D_jU_i+κD_i(i, j),

because D_iD_j =D_jD_i. Supposei > j then U_iD_j =D_ix_iD_j−κ X

s<i,s6=j

(i, s)D_j−κ(i, j)D_j

=D_i(D_jxi+κ(i, j))−κD_j X

s<i, s6=j

(i, s)−κD_i(i, j) =D_jU_i+κD_j(i, j).

Proposition 6.5. The spectral vector of θ_mD_mζ_α,T equals ξ

α,Tb . Proof . For the rankr

αb consider

rbα(1) = #{j: 1≤j < m, α_j > α_m−1}+ 1 = #{j: 1≤j < m, α_j ≥α_m}+ 1 =r_α(m), and for 1< i≤m

rbα(i) = #{j: 2≤j≤i, αj−1 ≥αi−1}+ #{j:j > i, αj−1> αi−1}+c_i,

where c_i = 1 if α_m −1 ≥ αi−1 equivalently if α_m > αi−1 and c_i = 0 otherwise, thus r

αb(i) = rα(i−1). Apply (6.1) withi=m toD_mζα,T to obtain

D_mxmD_mζα,T =D_m D_mxm−1−κX

i<m

(i, m)−κX

i>m

(i, m)

! ζα,T

=D_m(U_m−1)ζα,T −κX

i>m

(i, m)D_iζα,T = (ξα,T(m)−1)D_mζα,T,