TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 356, Number 1, Pages 159184 S 0002-9947(03)03037-X Article electronically published on August

26, 2003

CLOSED PRODUCT FORMULAS FOR EXTENSIONS OF GENERALIZED VERMA MODULES

RICCARDO BIAGIOLI

Abstract. We give explicit combinatorial product formulas for the polynomials encoding the dimensions of the spaces of extensions of (g, p)-generalized Verma modules, in the cases when (g, p) corresponds to an indecomposable classic Hermitian symmetric pair. The formulas imply that these dimensions are combinatorial invariants. We also discuss how these polynomials, dened by Shelton, are related to the parabolic R-polynomials introduced by Deodhar.

1. Introduction Let g be a complex, semisimple Lie algebra with Cartan subalgebra h and let b be a Borel subalgebra of g with nilradical n and Levi component h. Let p be a parabolic subalgebra of g containing b with nilradical u n and Levi component m h. So we have Levi decompositions b = h n and p = m u. The problem of computing the u-cohomology of irreducible highest weight (g, p)-modules is completely solved by the Kazhdan-Lusztig conjectures ([7]). On the other hand, the Extension Problem, namely the problem of computing the u-cohomology of the (g, p)-generalized Verma modules, is as yet unsolved in general, and there are not even conjectures for the form of the answer. If p = b, the modules in question are the Verma modules (see [9]). There was a conjecture that solved the problem, known as the Gabber-Joseph conjecture (see, e.g., [14] or [8]), but this was disproved by Boe in [2]. In the cases when (g, p) corresponds to an indecomposable Hermitian symmetric pair (see, e.g., [9, 4]), Shelton gives a solution to this problem. His answer is in terms of a recursion relations for the polynomials encoding the dimensions of the spaces of extensions between generalized Verma modules [17]. In this paper we solve these recursion relations, in the cases when (g, p) is an indecomposable classic Hermitian symmetric pair (see Table 1 at the end of 2). Our main results are explicit combinatorial product formulas for these polynomials. These formulas can be stated in two dierent ways, one in terms of Weyl group elements and one in terms of partitions or shifted partitions. Moreover, these formulas imply that these polynomials are combinatorial invariants. The organization of the paper is as follows. In the next section we collect some denitions and results that are needed in the rest of the work. In 3 we study the case of the symmetric pair (An1 , Ai1 Ani1 ), in 4 the case of (Cn , An1 ), in 5 the case of (Dn , An1 ), and in 6 the remaining cases (Bn , Bn1 ) and (Dn , Dn1 ). In the rst three cases we give two statements of these results, one in terms of
Received by the editors September 24, 2001 and, in revised form, February 11, 2002. 2000 Mathematics Subject Classication. Primary 17B10, 05E99; Secondary 22E47, 20F55.
c 2003 American Mathematical Society

159

160

RICCARDO BIAGIOLI

permutations or signed permutations and one in terms of partitions or shifted partitions. In 7 we derive some consequences of our results and in particular we prove that these polynomials are combinatorial invariants. We also discuss how they are related to the parabolic R-polynomials introduced in [12], and the pleasant symmetry that follows with the Gabber-Joseph conjecture [14]. 2. Notation, definitions, and preliminaries In this section we give some denitions, notation and results that will be used in the rest of this work. We let P := {1, 2, 3, . . . }, N := P {0}, and Z be the set of integers; for a N we let [a] := {1, 2, . . . , a} (where [0] := ) and [n] := [n, n] \ {0}. The cardinality of a set A will be denoted by |A|. For any sequence (a1 , . . . , an ) Zn , we let N1 (a1 , . . . , an ) := |{i [n] : ai < 0}|. Let Sn be the set of all bijections : [n] [n]. If Sn , then we write = 1 . . . n to mean that (i) = i , for i = 1, . . . , n. If Sn , then we will also write in disjoint cycle form (see, e.g., [18], p.17) and we will usually omit writing the 1-cycles of . For example, if = 64175823, then we also write = (2, 4, 7)(1, 6, 8, 3). Given , Sn we let := (composition of functions) so that, for example, (1, 2)(2, 3) = (1, 2, 3). We follow Chapter 3 of [18] for poset notation and terminology. In particular, given a poset (P, ) and u, v P we let [u, v ] := {z P : u z v } and call this an interval of P . If P has a minimal element, denoted 0, then we call a subset of the form [ 0, u], for u P , a lower interval of P . Given any Q P we will always consider Q as a poset with the partial ordering induced by P and call Q a subposet of P . We say that z P is join-irreducible if it covers at most one element of P . Given two posets P and Q we write P = Q to mean that they are isomorphic as posets. We follow 7.2 of [19] for any undened notation and terminology concerning partitions. By an (integer) partition we mean a sequence of positive integers = i . The number (1 , . . . , k ) such that 1 2 . . . k and we write || = of parts of is the length of , denoted l(). We identify a partition with its diagram, {(i, j ) P2 : 1 i k, 1 j i }. If we replace the dots (i, j ) by juxtaposed squares, we obtain the Young diagram of , and we draw it rotated counterclockwise by 4 radians. So, for example, the diagram of (5, 4, 2, 1) is illustrated in Figure 1. We call the elements of P2 , and hence of , cells. Expressions such as to the left of, or directly above, always refer to these rotated diagrams. We denote by P the set of all integer partitions. For any , P we dene if and only

Figure 1.

CLOSED PRODUCT FORMULAS FOR VERMA MODULES

161

if i i for all i. If we identify a partition with its (Young) diagram, then the partial order is given simply by containment of diagrams. It is well known, and not hard to see, that this makes P into a lattice, usually called Youngs lattice (see, e.g., [19, 7.2]). Given n P and i [n 1] we let P (n, i) := { P : (n i)i }. Given = (1 , . . . , k ) P we let d() be the length of the Durfee square of , so (1) d() := max {i [k ] : i i}.

Let , P , . We then call \ a skew partition. Note that, in poset theoretic language, partitions (respectively, skew-partitions) are the nite order ideals (respectively, nite convex subsets) of P2 . Given two skew partitions , P2 we write if is a translate of . Given a skew partition P2 its conjugate is := {(j, i) P2 : (i, j ) }. We follow [15] for general Coxeter groups notation. In particular, given a Coxeter system (W, S ) and W we denote by l( ) the length of in W , with respect to S , and we let D( ) := {s S : l(s) < l( )}. We denote by e the identity of W , and we let T := {s 1 : W, s S } be the set of reections of W . Given J S we let WJ be the parabolic subgroup generated by J and W J := { W : l(s ) > l( ) for all s J }. Note that W = W . The quotient W J is a poset and it is partially ordered by (strong) Bruhat order. Recall (see, e.g., [15, 5.9]) that this means that x y if and only if there exist r N and t1 , . . . , tr T such that tr . . . t1 x = y and l(ti . . . t1 x) > l(ti1 . . . t1 x) for i = 1, . . . , r. In this order, e is the minimal element J 0 and wJ denote the unique maximal elements in W J and WJ , and, if W is nite, w0 respectively. Given u, v W J , u v , we let [u, v ]J := {z W J : u z v }, and consider [u, v ]J as a poset with the partial ordering induced by W J . In the remainder of this section, we recall a result of Shelton, for which we need some notation from [13] and [17]. Let g be a complex, semisimple Lie algebra with Cartan subalgebra h, and let b be a Borel subalgebra of g with nilradical n and Levi component h. Let p be a parabolic subalgebra of g with nilradical u and Levi component m. Let W and WJ be the Weyl groups of g and m, respectively. Let , (m) be the root systems of g and m, with positive systems + and + (m), determined by n. Let 2 be the sum of the positive roots in g and x a + -antidominant integral weight h . For any h we denote by M () the g -Verma module with highest weight . Then L() and N () denote the unique irreducible quotient and the maximal m-locally nite quotient of M (), respectively. If is + (m)-dominant regular and integral, then N () is the (g, p)generalized Verma module of highest weight . For u W and w W J we 0 w). All Ext denote by Mu the module M (u) and by Nw the module N (wJ groups will be computed in Om , the usual Bernstein-Gelfand-Gelfand category of g -modules relative to the pair (g, p) (see, e.g., [1]). The following result is due to Shelton, and we refer the reader to [17] for its proof. He denes for any u and v in

162

RICCARDO BIAGIOLI

W J , a polynomial Eu,v (q ) by (2) and proves that Theorem 2.1. Suppose that (g, p) corresponds to an indecomposable Hermitian symmetric pair. Then for all u, v W J : i) Eu,v (q ) = 0 if u v ; ii) Eu,u (q ) = 1; iii) if u < v and s D(u) with us W J , then Eus,vs (q ) if s D(v ) and vs W J ; if s D(v ) and us vs; (q 1)Eus,v (q ) Eu,v (q ) = 1 ) E ( q ) + E ( q ) if u us vs v ; ( q q us,v us,vs if vs W J . qEus,v (q ) In this paper we solve these recurrence relations in the cases when (g, p) is an indecomposable classic Hermitian symmetric pair (see Table 1). Our main results are explicit product formulas for these polynomials. Moreover, these formulas imply that the polynomials are combinatorial invariants. Table 1. The indecomposable classic Hermitian Symmetric pairs (g, p). (g, p) g SU (r, s) AN SO(2n 1, 2) Bn Sp(2n, R) Cn SO(2n 2, 2) Dn SO (2n) Dn [m, m] Ar1 As1 Bn1 An1 Dn1 An1 Eu,v (q ) = (1)l(v)l(u)k q k dim(Ext (Nu , Nv )),
k k 0

3. The case (An1 , Ai1 Ani1 ) In this section, we give an explicit product formula for the polynomials Eu,v (q ) in the case of the pair (An1 , Ai1 Ani1 ). We give two dierent formulations of this result, one in terms of permutations and one in terms of partitions. Throughout this section we x n P and i [n 1], and we let W := Sn , si := (i, i + 1) for i = 1, . . . , n 1, S := {s1 , . . . , sn1 }, and J := S \ {si }. It is well known that (Sn , S ) is a Coxeter system of type An1 and that the following characterization holds (see, e.g., [15]). Proposition 3.1. Let v Sn . Then D(v ) = {si S : v (i) > v (i + 1)}. For symmetric groups, the parabolic subgroups of Sn are called Young subgroups. In the case of maximal parabolic subgroups (i.e. J = S \ {si }) their quotients take this particularly simple form: W J = {w W : w1 (1) < . . . < w1 (i) and w1 (i + 1) < . . . < w1 (n)}. Given v W J we associate to v the partition (3) (v ) := (v 1 (i) i, . . . , v 1 (2) 2, v 1 (1) 1).

CLOSED PRODUCT FORMULAS FOR VERMA MODULES

163

Figure 2. The lattice path corresponding to (5, 4, 2, 1) if n = 9 and i = 4.

Figure 3. The lattice path corresponding to (5, 4, 2, 1) if n = 12 and i = 5. The following is well known (see [4, Proposition 2.8]). Proposition 3.2. The map dened by (3) is a bijection between W J and P (n, i). Furthermore u v in W J if and only if (u) (v ), and l(v ) = |(v )| for all u, v W J . We will nd it sometimes convenient to identify a partition P (n, i) with a lattice path, with (1, 1) and (1, 1) steps. This path is the union of the lower boundary of the diagram of the skew-partition (n i)i \ and the upper boundary of the partition . Note that it starts at (0, 0) and ends at (n, 2i n) (equivalently, it has n steps and exactly i are (1, 1)-steps). We call a (1, 1)-step (respectively, (1, 1)-step) an up-step (respectively, down-step). Given j [n 1] we say that has a peak at j if the j -th step of is up and its (j +1)-th step is down. For example, if we take the partition of Figure 1, = (5, 4, 2, 1) P (9, 4), then the associated path is the one shown in Figure 2 and it has peaks at 1, 3, 6, and 8. Note that this identication between partitions and paths depends on n and i. For example, the same partition (5, 4, 2, 1) corresponds to the path in Figure 3 if n = 12 and i = 5. Since n and i are xed throughout this section, this will cause no confusion. The following elementary lemma is known (see [4, Lemma 2.9]). Lemma 3.3. Let v W J , and j [n 1]. Then sj D(v ) if and only if (v ) has a peak at n j . Note that the k -th step of (v ) is an up step if and only if k {n + 1 v 1 (i), n + 1 v 1 (i 1), . . . , n + 1 v 1 (1)}. Let u, v W J , u v . For j [n] we let, following [4], (4) (5) aj (u, v ) := |{r u1 ([i]) : r < j }| |{r v 1 ([i]) : r < j }|. (a1 (u, v ), . . . , a9 (u, v )) = (0, 1, 1, 2, 1, 1, 2, 1, 1). For example, if n = 9, i = 4, u = 123564789, and v = 516278394, then Note that it follows easily from Proposition 3.2 that aj (u, v ) 0 for j = 1, . . . , n if and only if u v , and that aj (u, v ) > 0 if j v 1 ([i]) \ u1 ([i]) and u v . Also note that, if u W J and j [n], then |{r u1 ([i]) : r < j }| = u(j ) 1, if j u1 ([i]), j + i u(j ), if j u1 ([i]).

164

RICCARDO BIAGIOLI

This may be used to obtain a more explicit formula for aj (u, v ), if desired. We can now state and prove the main result of this section. Theorem 3.4. Let u, v W J , u v . Then (6) Eu,v (q ) = q l(v)l(u)
j v 1 ([i])\u1 ([i])

(1 q 2aj (u,v)+1 ).

Proof. Let, for brevity, Di (u, v ) := v 1 ([i]) \ u1 ([i]). We proceed by induction J J J ) l(u). If l(w0 ) l(u) = 0, we have w0 = v = u and the result is on l(w0 J trivially true. So suppose that l(w0 ) l(u) > 0 and let s = (k, k + 1) be such that s D(u) and us W J . Note that, since u W J , this implies that k u1 ([i]) and k + 1 u1 ([i]). We have four cases to consider. a) s D(v ), and vs W J . Since v W J , this implies that k v 1 ([i]) and k + 1 v 1 ([i]). Moreover, us, vs W J and so Di (us, vs) = Di (u, v ) and aj (us, vs) = aj (u, v ) for all j [n], since (us)1 ([i]) is obtained from u1 ([i]) by replacing k by k + 1, and similarly for v . Hence, by Theorem 2.1 and our induction hypothesis, Eu,v (q ) = = = Eus,vs q l(vs)l(us)
j Di (us,vs)

(1 q 2aj (us,vs)+1 ) (1 q 2aj (u,v)+1 ),

q l(v)l(u)
j Di (u,v )

as desired. b) s D(v ), and us vs. Since v W J this implies that k v 1 ([i]), k + 1 v 1 ([i]), and vs W J . So we have that Di (us, v ) = Di (u, v ) \ {k + 1}, (7) aj (us, v ) = aj (u, v ) = aj (us, vs)

for all j [n] \ {k + 1}, and (8) ak+1 (us, vs) = ak+1 (u, v ) 2.

Since us vs by (8) and the note before the statement of this theorem, we conclude that ak+1 (u, v ) = 1. Hence, by Theorem 2.1 and the induction hypothesis, Eu,v (q ) = = (q 1)Eus,v (q ) (q 1)q l(v)l(u)1
j Di (u,v )\{k+1}

(1 q 2aj (us,v)+1 ) (1 q 2aj (u,v)+1 )

j Di (u,v )

= = as desired.

(q 1)q (1 q 2ak+1 (u,v)+1 ) q

l(v )l(u) j Di (u,v )

l(v )l(u)1

(1 q 2aj (u,v)+1 ),

CLOSED PRODUCT FORMULAS FOR VERMA MODULES

165

c) s D(v ), and us vs. Since v W J this implies that k v 1 ([i]), k + 1 v 1 ([i]), and vs W J . So we have that Di (us, v ) = Di (u, v ) \ {k + 1}, Di (us, vs) = (Di (u, v ) \ {k + 1}) {k }, aj (us, v ) = aj (us, vs) = aj (u, v ) for all j [n] \{k +1}, ak+1 (us, vs) = ak+1 (u, v ) 2 and ak+1 (u, v ) = ak (u, v )+1. Hence, by Theorem 2.1 and our induction hypothesis, Eu,v (q ) = = (q q 1 )Eus,v (q ) + Eus,vs (q ) (q q 1 )q l(v)l(u)1
j Di (u,v )\{k+1}

(1 q 2aj (us,v)+1 ) (1 q 2aj (us,vs)+1 )

+ q l(v)l(u)2
j (Di (u,v )\{k+1}){k}

(q q +q

)q

l(v )l(u)1 j Di (u,v )\{k+1}

(1 q 2aj (u,v)+1 ) )
j Di (u,v )\{k+1}

l(v )l(u)2

(1 q
2

2ak (u,v )+1

(1 q 2aj (u,v)+1 ) (1 q 2aj (u,v)+1 )

j Di (u,v )

= =

q l(v)l(u)2 q
l(v )l(u)

(q q ) 2 a ( u,v )+1 k +1 (1 q ) (1 q

2ak (u,v )+1

2aj (u,v )+1

),

j Di (u,v )

and the result again follows. d) vs W J . Then s D(v ) and we have two cases. In the rst one, we have k, k + 1 v 1 ([i]), which implies that Di (us, v ) = (Di (u, v ) \ {k + 1}) {k } and aj (us, v ) = aj (u, v ) for j [n] \ {k + 1}, ak+1 (u, v ) = ak (us, v ). Hence, by Theorem 2.1 and the induction hypothesis, Eu,v (q ) = = qEus,v (q ) q l(v)l(u)
j (Di (u,v )\{k+1}){k}

(1 q 2aj (us,v)+1 ) (1 q 2ak (us,v)+1 ) (1 q 2ak+1 (u,v)+1 )

j Di (u,v )

= =

q l(v)l(u) q l(v)l(u)

(1 q 2aj (u,v)+1 )
j Di (u,v )

(1 q 2aj (u,v)+1 ).

In the second case we have, k, k + 1 v 1 ([i]), hence Di (us, v ) = Di (u, v ) and aj (us, v ) = aj (u, v ) for j [n] \ {k + 1}. Hence, by Theorem 2.1 and the induction

166

RICCARDO BIAGIOLI

hypothesis, Eu,v (q ) = qEus,v (q ) = q l(v)l(u)

j Di (u,v )

(1 q 2aj (us,v)+1 ) (1 q 2aj (u,v)+1 ),

j Di (u,v )

= q l(v)l(u)

and the result again follows. This completes the induction step and hence the proof. Because W J is isomorphic, as a poset, to a lower interval in Youngs lattice (by Proposition 3.2), it is natural to rephrase Theorem 3.4 in the language of partitions rather than permutations. Let , P (n, i), with . Think of and as paths as explained above. Then, by Proposition 3.2, the path lies (weakly) above the path . Let 1 j n and consider the j -th step of (from the left). Following [4], we say that such a step is allowable with respect to if it is an up-step and the j -th step of is a down-step. For example, if n = 9, i = 4, = (5, 4, 2, 1), and = (2, 0, 0, 0), then the j -th step of is allowable with respect to exactly if j {1, 3, 6} (see Figure 4). Now let a j (, ) be the vertical distance (divided by two, since it is always even) between the (right end of the) j -th step of and the (right end of the) j -th step of . We then have the following result, and we refer the reader to [4, Proposition 3.3] for its proof. Proposition 3.5. Let u, v W J , u v . Then n+1j ((u), (v )) aj (u, v ) = a for j = 1, . . . , n. Furthermore n + 1 j v 1 ([i]) \ u1 ([i]) if and only if the j -th step of (v ) is allowable with respect to (u). We can now rephrase Theorem 3.4 in terms of partitions. Corollary 3.6. Let u, v W J , u v . Then (9) Eu,v (q ) = q |\|
j aj (,)+1 (1 q 2 ),

where = (u), = (v ) and j runs over all the allowable steps of with respect to . In particular, Eu,v (q ) depends only on (v ) \ (u). In the case of a lower interval, formula (9) takes up a particularly simple form. The proof of the next result is analogous to the one of [4, Corollary 3.5], and we leave it to the reader. Corollary 3.7. Let v W J . Then
d()

Ee,v (q ) = q ||
j =1

(1 q 2j +1 ),

where = (v ) and d() is the length of the Durfee square of .

CLOSED PRODUCT FORMULAS FOR VERMA MODULES

167

Figure 4.

Figure 5. We close this section with an example to illustrate Theorem 3.4 and Corollary 3.6. We use the various calculations made in 2 and 3. Let u = 123564789 and S \{(4,5)} ; we have D4 (u, v ) = {4, 7, 9}. From (5) and (6), it v = 516278394 in S9 follows that (10) Eu,v (q ) = q 10 (1 q 1 )(1 q 3 )(1 q 3 ). Observe that we have (v ) = (5, 4, 2, 1) = and (u) = (2, 0, 0, 0) = . The diagram of the skew-partition (v ) \ (u) is drawn in Figure 4. The allowable are 3 (, ) = 2, a 6 (, ) = 2. indicated by arrows and a 1 (, ) = 1, a 4. The case (Cn , An1 ) A partition (1 , 2 , . . . , k ) is strict if 1 > 2 > . . . > k . We denote by Ps the set of all (integer) strict partitions. Let H := {(i, j ) P2 : i j } with the ordering induced by the product ordering on P2 . We call the nite order ideals of H shifted partitions. Denote by I the set of all nite order ideals of H . Note that I is partially ordered by set inclusion. It is well known that this makes I into a distributive lattice. We identify a shifted partition with its diagram {(i, j ) P2 : 1 i k, i j i 1 + i}, and as in 3 we draw it rotated counterclockwise by diagram of (7, 6, 5, 4, 2) is illustrated in Figure 5.
4

radians. So for example the

168

RICCARDO BIAGIOLI

Let := { P : l()l() }. P Note that there are inclusion preserving bijections between strict partitions, shifted , given by partitions and partitions in P (1 , 2 , . . . , k ) {(i, j ) P2 : 1 i k, i j i 1 + i} (1 , 2 + 1, . . . , k + k 1). For this reason, we will freely identify these objects. Note, however, that only the bijection between strict partitions and shifted partitions preserves size. In fact, if and P , with , then it does not necessarily follow that P . , , P Therefore the subposet { P : } of P is not isomorphic to the subposet { I : } of I . Our purpose in this section is to obtain an explicit product formula for the E B be the group of polynomials in the case of the pair (Cn , An1 ). Therefore, let Sn all the bijections of the set [n] in itself such that w(a) = w(a)
B , then, following for all a [n], with composition as group operation. If w Sn [6], we write w = [a1 , . . . , an ] to mean that w(i) = ai , for i = 1, . . . , n, and call this B is often called the the window notation of w. Because of this notation the group Sn B are permutations group of all signed permutations on [n]. Since the elements of Sn of [n] we can also write them in disjoint cycle form; as in 2, we multiply elements B in the natural way. For the from the right. We identify Sn as a subgroup of Sn B , si := (i, i + 1)(i 1, i) rest of this section, we x n P and we let W := Sn for i = 1, . . . , n 1, s0 := (1, 1), S := {s0 , s1 , . . . , sn1 } and J := S \ {s0 }. It B , S ) is a Coxeter system of type Bn and that the following is well known that (Sn characterization holds (see, e.g., [6]). B . Then Proposition 4.1. Let v Sn

D(v ) = {si S : v (i) > v (i + 1)}, where v (0) := 0. From Proposition 4.1 we have that W J = {v W : v 1 (1) < v 1 (2) < . . . < v 1 (n)}. Therefore, given v = e, v W J , there is a unique k [n] (in fact, k = N1 (v )) such that (11) (12) Let (n) := { I : (n, n 1, . . . , 2, 1)}. I v 1 (k ) < 0 < v 1 (k + 1) B (v ) := (v 1 (1), v 1 (2), . . . , v 1 (k )). and we associate to v the shifted partition

CLOSED PRODUCT FORMULAS FOR VERMA MODULES

169

Figure 6. The following is known: Proposition 4.2. The map B dened by (12) is a bijection between W J and (n). Furthermore u v in W J if and only if B (u) B (v ) and l(v ) = |B (v )| I for all u, v W J . (n) with a lattice As before it is convenient to identify a shifted partition I path with (1, 1) and (1, 1) steps starting at (0, 0) and having n steps. We have the obvious bijection between the peaks of as a path and the upper peaks of as a partition. Note that, as in Proposition 3.2, this bijection depends on n, but for us n is xed and so there is no confusion. For example, the partition = (7, 6, 5, 4, 2) corresponds to the path in Figure 6. Lemma 4.3. Let v W J and j [n 1]. Then sj D(v ) if and only if B (v ) has a peak at n j . Furthermore, so D(v ) if and only if the last step of B (v ) is up. This result can be proved in a way similar to Lemma 3.3 (see also Lemma 5.3) and is due to Brenti [5]. Note that i-th step of B (v ) is an up-step if and only if (13) i {n + 1 + v 1 (1), n + 1 + v 1 (2), . . . , n + 1 + v 1 (k )}. Proposition 4.4. Let v W J and i [n]. Then the i-th step (from the left) of B (v ) (seen as a path) is an up-step if and only if v (n + 1 i) < 0. Proof. We know that the i-th step of v W J is an up-step if and only if (14) n + 1 i {v 1 (1), v 1 (2), . . . , v 1 (k )}. But this, by the denition of k , happens if and only if v (n +1 i) < 0, as desired. We are now ready to prove the main theorem of this section, which gives an explicit product formula for the polynomials Eu,v (q ) in the case of the symmetric pair (Cn , An1 ). As in the preceding section we give two dierent formulations of this result, one in terms of signed permutations and one in terms of shifted partitions. Let u, v W J , u v . For j [n] let, following [5], (15) (16) bj (u, v ) := |{r j : v (r) < 0}| |{r j : u(r) < 0}|. (b1 (u, v ), . . . , b7 (u, v )) = (2, 2, 1, 2, 2, 1, 0). For example, if u = [4, 5, 3, 2, 6, 7, 1] and v = [6, 5, 7, 4, 3, 2, 1], then Note that it follows from Proposition 4.2 that bj (u, v ) 0 for j = 1, . . . , n if and only if u v . Also, if u v , then bj (u, v ) > 0 when v (j ) < 0 < u(j ). We let (17) N (u, v ) := {r [n] : u(r)v (r) < 0}

170

RICCARDO BIAGIOLI

and (18) D(u, v ) := {r N (u, v ) : (1)br (u,v) < 0}.

Theorem 4.5. Let u, v W J , u v . Then (19) Eu,v (q ) = q l(v)l(u)

j D(u,v )

(1 q bj (u,v) ),

where (20) bj (u, v ) := 2bj (u, v ) 1 if u(j ) > 0, 2bj (u, v ) + 1 if u(j ) < 0.

J J ) l(u). If l(w0 ) l(u) = 0, we have Proof. We proceed by induction on l(w0 J J ) l(u) > 0 and w0 = v = u and the result is trivially true. So suppose that l(w0 J let s be such that s D(u) and us W . Suppose rst that s = (i1, i)(i, i+1) for some i [n1]; then u(i) < u(i+1). Note that, since u W J , this implies that u(i) < 0 < u(i + 1). We have four cases to consider.

a) s D(v ), and vs W J . Since v W J , this implies that v (i) < 0 < v (i + 1). Moreover us, vs W J , and so N (us, vs) = N (u, v ) and bj (us, vs) = bj (u, v ) for all j [n], so that D(us, vs) = bj (u, v ) for all j [n] \ {i, i + 1}. Hence, by Theorem 2.1 D(u, v ) and bj (us, vs) = and our induction hypothesis, Eu,v (q ) = Eus,vs (q ) = q l(vs)l(us)
j D(us,vs)

(1 q bj (us,vs) ) (1 q bj (u,v) ),

= q l(v)l(u)
j D(u,v )

as desired. b) s D(v ), and us vs. Then v (i) > 0 > v (i + 1), therefore N (us, v ) = N (u, v ) \ {i, i + 1}, bj (us, v ) = bj (u, v ) = bj (us, vs) for all j [n] \ {i + 1}, and (21) bi (u, v ) = bi+1 (u, v ) 1.

Since us vs, it follows from the note before the statement of the theorem that bi+1 (u, v ) > 0 > bi+1 (us, vs). Also bi+1 (u, v ) bi+1 (us, vs) = 2, so bi+1 (u, v ) = 1. This and (21) imply that i D(u, v ) and i + 1 D(u, v ). It follows that bj (u, v ) for all j [n] \ {i, i + 1}, and D(us, v ) = D(u, v ) \ {i + 1}, bj (us, v ) = that bi+1 (u, v ) = 2bi+1 (u, v ) 1 = 1. Hence, by Theorem 2.1 and the induction

CLOSED PRODUCT FORMULAS FOR VERMA MODULES

171

hypothesis, Eu,v (q ) = (q 1)Eus,v (q ) = (q 1)q l(v)l(u)1

j D(u,v )\{i+1}

(1 q bj (us,v) ) (1 q bj (u,v) )
j D(u,v )

(q 1)q (1

l(v )l(u)1

bi+1 (u,v ) ) q

= q

l(v )l(u) j D(u,v )

(1 q bj (u,v) ).

c) s D(v ), and us vs. Then, as above, v (i) > 0 > v (i + 1), N (us, v ) = N (u, v ) \ {i, i + 1}, bj (us, v ) = bj (u, v ) for all j [n] \ {i + 1}, and (22) bi (u, v ) = bi+1 (u, v ) 1.

bj (u, v ), for all j It follows that D(us, v ) = D(u, v ) \ {i, i + 1} and bj (us, v ) = [n] \ {i, i + 1}. On the other hand, we have N (us, vs) = N (u, v ) and bj (us, vs) = bj (u, v ) for all j [n] \ {i + 1}, and bi+1 (us, vs) = bi+1 (u, v ) 2. It follows that bj (u, v ) for all j [n] \ {i, i + 1}. Hence, by D(us, vs) = D(u, v ) and bj (us, vs) = Theorem 2.1 and the induction hypothesis, Eu,v (q ) = = (23) (q q 1 )Eus,v (q ) + Eus,vs (q ) (q q 1 )q l(v)l(u)1
j D(u,v )\{i,i+1}

(1 q bj (us,v) ) (1 q bj (us,vs) ).

+ q l(v)l(u)2
j D(u,v )

From (22) we have that i D(u, v ) if and only if i + 1 D(u, v ), so we have two cases. If i D(u, v ), then D(us, v ) = D(u, v ) \{i} and bi (us, vs) = 2bi (us, vs) 1 = bi (u, v ) 2 because u(i) < 0 < u(i + 1). Hence, by (23), 2bi (u, v ) 1 = Eu,v (q ) = (q 2 1)q l(v)l(u)2
j D(u,v )\{i}

(1 q bj (u,v) )

+ q l(v)l(u)2 (1 q bi (us,vs) )
j D(u,v )\{i}

(1 q bj (u,v) ) (1 q bj (u,v) )

= =

q l(v)l(u)2 q
l(v )l(u)

(q q
2

bi (us,vs)

)
j D(u,v )

(1

bi (u,v ) ) q

(1 q
j D(u,v )

bj (u,v )

).

172

RICCARDO BIAGIOLI

If i+1 D(u, v ), then D(us, v ) = D(u, v )\{i+1} and bi+1 (us, vs) = 2bi+1 (us, vs) bi+1 (u, v ) 2. Hence, by (23), + 1 = (2bi+1 (u, v ) 1) 2 = Eu,v (q ) = (q 2 1)q l(v)l(u)2
j D(u,v )\{i+1}

(1 q bj (u,v) )

+q

l(v )l(u)2

(1 q bi+1 (us,vs) )
j D(u,v )\{i+1} bi+1 (us,vs)

(1 q bj (u,v) ) (1 q bj (u,v) )

= =

q l(v)l(u)2 q
l(v )l(u)

(q q
2

)
j D(u,v )

(1

bi+1 (u,v ) ) q bj (u,v )

(1 q
j D(u,v )

),

and the result follows also in this case. d) vs W J . Then s D(v ) and we have two cases. In the rst one, we have v (i) < v (i + 1) < 0 and this implies that N (us, v ) = (N (u, v ) \ {i + 1}) {i}, bj (us, v ) = bj (u, v ) for all j [n] \ {i + 1}, bi (us, v ) = bi+1 (u, v ). It follows that D(us, v ) \ {i} = D(u, v ) \ {i + 1}, bj (u, v ) for all j [n] \{i, i +1} i D(us, v ), if and only if i +1 D(u, v ), bj (us, v ) = and bi (us, v ) = bi+1 (u, v ). Hence, by Theorem 2.1 and the induction hypothesis, if i D(us, v ), D(us, v ) = D(u, v ) and the thesis easily follows. Otherwise if i D(us, v ), we have Eu,v (q ) = = qEus,v (q ) qq l(v)l(u)1
j D(us,v )

(1 q bj (us,v) ) (1 q bj (u,v) )

= =

q l(v)l(u) q l(v)l(u)

(1 q bi (us,v) )
bi+1 (u,v ) ) (1 q j D(u,v )

(1 q bj (u,v) ),
j D(u,v )

bi+1 (u, v ). because bi (us, v ) = In the second case, we have 0 < v (i) < v (i + 1) and this implies that N (us, v ) = (N (u, v ) \ {i}) {i + 1}, bj (us, v ) = bj (u, v ) for all j [n] \ {i + 1}, bi+1 (us, v ) = bi (u, v ). It follows that D(us, v ) \ {i + 1} = D(u, v ) \ {i}, bj (u, v ) for all j [n] \{i, i +1} i +1 D(us, v ), if and only if i D(u, v ), bj (us, v ) = and bi+1 (us, v ) = bi (u, v ). Hence, by Theorem 2.1 and the induction hypothesis, if i + 1 D(us, v ), D(us, v ) = D(u, v ) and the thesis easily follows. Otherwise if

CLOSED PRODUCT FORMULAS FOR VERMA MODULES

173

i + 1 D(us, v ), we have Eu,v (q ) = = qEus,v (q ) q l(v)l(u)

j D(us,v )

(1 q bj (us,v) ) (1 q bi+1 (us,v) )

bi (u,v ) ) (1 q j D(u,v )

q l(v)l(u)

(1 q bj (u,v) ),

bi (u, v ), the result again follows. since bi+1 (us, v ) = Suppose now that s = (1, 1) = s0 . Then u(1) > 0, and we observe that us, vs W J . We therefore have three cases to consider. 1) s D(v ). Then vs W J and v (1) > 0. Hence N (us, vs) = N (u, v ), bj (us, vs) = bj (u, v ) for bj (u, v ), for all all j [n] and so D(us, vs) = D(u, v ) , 1 D(u, v ), bj (us, vs) = j [n] \ {1} and the result follows from Theorem 2.1 and the induction hypothesis. 2) s D(v ), and us vs. Then v (1) < 0 so N (us, v ) = N (u, v ) \ {1}, bj (us, v ) = bj (u, v ) for all j [n] \ {1}. bj (u, v ) for all j [n] \ {1}. Therefore D(us, v ) = D(u, v ) \ {1} and bj (us, v ) = Also, u < v and us vs, so by the remark before the statement of the theorem, b1 (u, v ) = 1. Hence, b1 (u, v ) = 1 and hence Eu,v (q ) = (q 1)Eus,v (q ) = (q 1)q l(v)l(u)1
j D(u,v )\{1}

(1 q bj (u,v) )

= q

l(v )l(u) j D(u,v )

(1 q bj (u,v) ).

3) s D(v ), and us vs. Then v (1) < 0, N (us, vs) = N (u, v ), N (us, v ) = N (u, v ) \ {1}, bj (us, vs) = bj (u, v ) = bj (us, v ) for all j [n] \ {1}, and b1 (us, vs) = b1 (u, v ) 2 = b1 (us, v ) 1. It follows that D(us, v ) = D(u, v ) \ {1} = D(us, vs) \ {1} bj (u, v ) = bj (us, v ) for all j [n] \ {1}. Hence, and bj (us, vs) = Eu,v (q ) = = (24) (q q 1 )Eus,v (q ) + Eus,vs (q ) (q q 1 )q l(v)l(u)1
j D(u,v )\{1}

(1 q bj (us,v) ) (1 q bj (us,vs) ).

+q

l(v )l(u)2 j D(us,vs)

174

RICCARDO BIAGIOLI

Now we have two cases. If 1 D(u, v ), then 1 D(us, vs) and b1 (us, vs) = b1 (u, v ) 2, so from (24), Eu,v (q ) = (q 2 1)q l(v)l(u)2
j D(u,v )\{1}

(1 q bj (u,v) )

+q

l(v )l(u)2

(1 q b1 (us,vs) )
j D(u,v )\{1} b1 (us,vs)2 b1 (u,v ) ) q

(1 q bj (u,v) ) (1 q bj (u,v) ).

= q l(v)l(u)

(1 q

)
j D(u,v )

(1

If 1 D(u, v ), then 1 D(us, vs) and from (24) we have that Eu,v (q ) = (q 2 1)q l(v)l(u)2
j D(u,v )

(1 q bj (u,v) ) (1 q bj (u,v) )

+ q l(v)l(u)2
j D(u,v )

= q

l(v )l(u) j D(u,v )

(1 q bj (u,v) )

and the result follows. This completes the induction step and hence the proof. As in the previous section it is natural to rephrase Theorem 4.5 in the language (n), with . We think of and as paths, as of shifted partitions. Let , I explained at the beginning of this section. Then, by Proposition 4.2, the path lies (weakly) above the path . Let j [n] and consider the j -th step of (from the left). Following [5] we say that such a step is B -allowable with respect to if the j -th step of is not parallel to it and a j (, ) is odd. For example, if = (7, 4, 3, 0, 0, 0, 0) and = (7, 6, 5, 4, 2, 0, 0), then the j -th step of is B -allowable with respect to exactly if j {2, 5} (see Figure 7). Proposition 4.6. Let u, v W J , u v . Then n+1j (B (u), B (v )) bj (u, v ) = a for i = 1, . . . , n. Furthermore n + 1 j D(u, v ) if and only if the j -th step of B (v ) is B -allowable with respect to B (u). This result can be proved in a way similar to Proposition 3.5 and is due Brenti [5]. We can now rephrase Theorem 4.5 in terms of shifted partitions. Corollary 4.7. Let u, v W J , u v . Then (25) Eu,v (q ) = q |\|
j j (,) (1 q a ),

where = B (u), = B (v ), j runs over all the B -allowable steps of with respect to , and a j (, ) := 2 aj (, ) 1 2 aj (, ) + 1 if the j -th step of is down, if the j -th step of is up.

In particular, Eu,v (q ) depends only on B (v ) \ B (u).

CLOSED PRODUCT FORMULAS FOR VERMA MODULES

175

Figure 7. In the case of a lower interval, the formulas (19) and (25) take up a particularly simple form. Corollary 4.8. Let v W J . Then
N1 (v)+1 2

Ee,v (q ) = q where = B (v ).

l( v ) j =1

(1 q

4j +3

)=q

||

l()+1 2

(1 q 4j +3 ),

j =1

Proof. We know from Theorem 4.5 that (26) Ee,v (q ) = q l(v)

j D(e,v )

(1 q bj (e,v) ).

Clearly, N (e, v ) = {r [n] : v (r) < 0}, bj (e, v ) = 2bj (e, v ) 1 for all j [n]. However, bj (e, v ) = |{r j : v (r) < 0}| for j [n], and D(e, v ) = {r N (e, v ) : br (e, v ) is odd}. Hence,
N1 (v)+1 2

Ee,v (q ) = q

l( v ) j =1

(1 q 4j +3 ),

as desired. From the denition we have that l(B (v )) = N1 (v ), so the second equation follows. We close this section with an example. Let n = 7, u = [4, 5, 3, 2, 6, 7, 1], B J ) . We have N (u, v ) = {2, 3, 5, 6} and and v = [6, 5, 7, 4, 3, 2, 1] (S7 D(u, v ) = {3, 6}. Since u(3) < 0 and u(6) > 0, from (20) and (19) it follows that Eu,v (q ) = q 10 (1 q 3 )(1 q 1 ). Observe that B (u) = (7, 4, 3, 0, 0, 0, 0) = and B (v ) = (7, 6, 5, 4, 2, 0, 0) = . The paths corresponding to B (v ) and B (u) are drawn in Figure 7, and the B 5 (, ) = 3. allowable steps are indicated by arrows. We have that a 2 (, ) = 1 and a 5. The case (Dn , An1 ) In this section we study the E -polynomials in the case of the pair (Dn , An1 ). D B , the subgroup of Sn consisting of all the signed permutations Hence we consider Sn having an even number of negative entries in their window notation. More precisely,
D B := {w Sn : N1 (w(1), . . . , w(n)) 0 (mod 2)}. Sn D , si := (i, i + 1) For the rest of this section, we x n P and we let W := Sn (i 1, i) for i = 1, . . . , n 1, s0 := (1, 2)(1, 2), S := {s0 , s1 , . . . , sn1 } and

176

RICCARDO BIAGIOLI

Figure 8.
D , S ) is a Coxeter system of type Dn and J := S \ {s0 }. It is well known that (Sn that the following characterization holds (see, e.g., [6]). D . Then Proposition 5.1. Let v Sn

D(v ) = {si S : v (i) > v (i + 1)}, where v (0) := v (2) and v (n + 1) := 0. As in 4 we have that W J = {v W : v 1 (1) < v 1 (2) < . . . < v 1 (n)} and for every v W J v = e, there is a unique k [n] such that (27) (28) v 1 (k ) < 0 < v 1 (k + 1) D (v ) := (v 1 (1) 1, v 1 (2) 1, . . . , v 1 (k ) 1). and we associate to v the shifted partition It is not so hard to see that Proposition 5.2. The map D dened by (28) is a bijection between W J and (n 1). Furthermore u v in W J if and only if D (u) D (v ) and l(v ) = I |D (v )| for all u, v W J . (n) consisting of all the shifted partitions with an even Let A be the subset of I number of entries dierent to zero. More precisely, (n) : k is even}. A := { = (1 , 2 , . . . , k ) I We call this set the even shifted partitions. Observe that B (W J ) = A. Since, for B J D J ) if and only if u v in (Sn ) , we have an inclusion u, v W J , u v in (Sn (n 1) and even shifted preserving bijection between the shifted partition in I (n). Thanks to this bijection we can identify I (n 1) with the partitions in I lattice path associated to I (n) as explained in 4 after Proposition 4.2. We observe that this lattice path starts at (0, 0), ends after n steps, and has an even D J ) . Then number of up-steps. For example, let v = [4, 5, 3, 6, 2, 1] (S6 D (v ) = (5, 4, 2, 0), B (v ) = (6, 5, 3, 1) and the lattice path associate to v is drawn in Figure 8. Lemma 5.3. Let v W J and j [n 1]. Then sj D(v ) if and only if D (v ) has a peak at n j . Furthermore, so D(v ) if and only if the last two steps of D (v ) are up. Proof. Let k be dened by (27) and j [n 1]; we have that sj D(v ) if and only if v (j ) > v (j + 1). Since v W J , this happens if and only if v (j ) > 0 > v (j + 1). Equivalently, this happens if and only if j v 1 ([n]) and j +1 v 1 ([n]). However, D (v ) (as a path) has a peak at n j if and only if its (n j )-th step is up and its (n j + 1)-th step is down. But the i-th step of D (v ) is an up-step if and only if (29) i {n + 1 + v 1 (1), n + 1 + v 1 (2), . . . , n + 1 + v 1 (k )}.

CLOSED PRODUCT FORMULAS FOR VERMA MODULES

177

Therefore D (v ) has a peak at n j if and only if (30) for some i [k ], and (31) j = v 1 (i) for all i [k ]. Equivalently, if and only if j v 1 ([k, 1]) and j +1 v 1 ([k, 1]), but by the denition of k , i v 1 ([k, 1]) if and only if i v 1 ([n, 1]), which is if and only if i v 1 ([n]), for all i [n]. The result follows. Now let so D(v ). This happens if and only if v (1) + v (2) < 0 and implies that v (1) < v (2) < 0. It follows that v 1 (k ) = 1 and v 1 (k 1) = 2, and that the n-th and the n 1-th steps of D (v ) are up. The following is exactly the analogue of Proposition 4.4 for permutation in D J ) . (Sn Proposition 5.4. Let v W J and i [n]. Then the i-th step (from the left) of D (v ) (seen as a path) is an up-step if and only if v (n + 1 i) < 0. Now we are ready to prove the analogue of Theorem 4.5 for the permutations in D J ) . The formula is exactly the same, but observe that the polynomials are not (Sn always the same; in fact the function length is dierent. Theorem 5.5. Let u, v W J , u v . Then (32) Eu,v (q ) = q l(v)l(u)
j D(u,v )

j = v 1 (i) + 1

(1 q bj (u,v) ),

where bj (u, v ) is dened as in Theorem 4.5.

J J ) l(u). If l(w0 ) l(u) = 0, we have Proof. We proceed by induction on l(w0 J J ) l(u) > 0 and w0 = v = u and the result is trivially true. So suppose that l(w0 J let s be such that s D(u) and us W . If s = (i 1, i)(i, i + 1) for some i [n 1], then the proof is exactly the same as for Theorem 4.5. So suppose that s = (1, 2)(1, 2) = s0 ; then we have u(1) + u(2) > 0. We observe that us W J implies that 0 < u(1) < u(2). Moreover we have that b1 (u, v ) is even for every u, v W J , and so 1 D(u, v ) . We have four cases to consider. a) s D(v ) and vs W J .

We have 0 < v (1) < v (2). Then N (us, vs) = N (u, v ) and bj (us, vs) = bj (u, v ) for bj (u, v ) for all j [n] \{1, 2}. all j [n] and so D(us, vs) = D(u, v ) and bj (us, vs) = Hence, by Theorem 2.1 and our induction hypothesis, Eu,v (q ) = = b) s D(v ) and us vs. Hence v (1)+v (2) < 0 and v (1) < v (2) < 0. We have that N (us, v ) = N (u, v )\{1, 2}, bj (us, v ) = bj (u, v ) for all j [n] \ {1, 2}. Using reasoning as in Theorem 4.5b), b1 (u, v ) > b2 (u, v ) 0, so b1 (u, v ) 2; b2 (us, vs) > b1 (us, vs) so b1 (us, vs) 2, and b1 (u, v ) b1 (us, vs) = 4 so b1 (u, v ) = 2 and b2 (u, v ) = 1. So D(us, v ) = Eus,vs (q ) q l(v)l(u)
j D(u,v )

(1 q bj (u,v) ).

178

RICCARDO BIAGIOLI

D(u, v ) \ {2} and bj (us, v ) = bj (u, v ) for all j [n] \ {1, 2} and b2 (u, v ) = 1. Hence, by Theorem 2.1 and the induction hypothesis, Eu,v (q ) = (q 1)Eus,v (q ) = (q 1)q l(v)l(u)1
j D(u,v )\{2}

(1 q bj (u,v) )

= q l(v)l(u)
j D(u,v )

(1 q bj (u,v) ).

c) s D(v ) and us vs. We have v (1) + v (2) < 0 and v (1) < v (2) < 0. So N (us, vs) = N (u, v ), N (us, v ) = N (u, v ) \ {1, 2}, bj (us, vs) = bj (u, v ) = bj (us, v ) for all j [n] \ {1, 2}, and b2 (us, vs) = b2 (u, v ) 2. It follows that D(us, v ) = D(u, v ) \ {2}, D(us, vs) = D(u, v ), bj (u, v ) = bj (us, v ) for all j [n] \ {1, 2}, and b2 (us, vs) = b2 (u, v ) 2. bj (us, vs) = Hence, by Theorem 2.1 and our induction hypothesis, Eu,v (q ) = = + (q q 1 )Eus,v (q ) + Eus,vs (q ) (q q 1 )q l(v)l(u)1
j D(u,v )\{2}

(1 q bj (u,v) ) )
j D(u,v )\{2}

l(v )l(u)2

(1 q

b2 (us,vs)

(1 q bj (u,v) ) (1 q bj (u,v) ),
j D(u,v )

q l(v)l(u)

(1 q

b2 (us,vs)2

b2 (u,v ) ) (1 q

and the result follows. d) vs W J . We have two cases. Suppose that v (1) > 0 and v (2) < 0. We have N (us, v ) = (N (u, v ) {1}) \ {2}, bj (u, v ) = bj (us, v ) for all j [n] \ {1, 2}, and (33) (34) b1 (u, v ) = b2 (u, v ), b1 (us, v ) = b1 (u, v ) 2.

Since b1 (u, v ) is even, (33) implies that 2 D(u, v ), and (34) implies that 1 bj (u, v ) for all j D(us, v ). It follows that D(u, v ) = D(us, v ) and bj (us, v ) = [n] \ {1, 2}, so the thesis follows immediately by induction. Suppose now that v (1) < 0 and v (2) > 0. Then N (us, v ) = (N (u, v ) {2}) \ {1}, bj (u, v ) = bj (us, v ) for all [n] \ {1, 2}, and (35) b1 (u, v ) = b2 (us, v ) + 2. Since b1 (u, v ) is even, (35) implies that 2 D(us, v ). It follows that D(u, v ) = bj (u, v ) for all j [n] \ {1, 2}, so the thesis follows immeD(us, v ) and bj (us, v ) = diately by induction. This completes the proof. As in the previous cases it is natural to rephrase Theorem 5.5 in the language of shifted partitions. Let , A, with . We think of and as paths as explained in 4. So using Proposition 4.6 we have that

CLOSED PRODUCT FORMULAS FOR VERMA MODULES

179

Corollary 5.6. Let u, v W J , u v . Then (36) Eu,v (q ) = q |\|

j j (,) (1 q a )

where = D (u), = D (v ), j runs over all the B -allowable steps of with respect to , and a j (u, v ) is dened as in Corollary 4.7. In particular, Eu,v (q ) depends only on D (v ) \ D (u). For lower intervals we obtain the following Corollary 5.7. Let v W J . Then
N1 (v) 2

Ee,v (q ) = q where = D (v ).

l( v ) j =1

(1 q

4j +3

)=q

||

l() 2

(1 q 4j +3 ),

j =1

Proof. The results follows immediately from Corollary 4.8 , observing that N1 (v ) is even for every v W J . 6. The cases (Bn , Bn1 ) and (Dn , Dn1 ) In this section we analyze the E -polynomials in the cases (Bn , Bn1 ) and (Dn , Dn1 ). We start with the rst one. We use the same notations as in 4, hence B , S = {s0 , s1 , . . . , sn1 }, and so = (1, 1), but now we let J := S \ {sn1 }, W = Sn so as to have WJ = Bn1 . It follows that the quotient W J is a totally ordered set, more precisely, is the chain W J = {e, sn1 , . . . , sn1 sn2 . . . s1 s0 , sn1 . . . s1 s0 s1 , . . . , sn1 sn2 . . . s1 s0 s1 . . . sn1 }. Proposition 6.1. Let (W, S ) be a Coxeter system, J S and u, v W J such that u v . If [u, v ]J is a chain, then Eu,v (q ) = q l(v)l(u) (1 q 1 ).
J J ) l(u). If l(w0 ) l(u) = 0, the result is Proof. We proceed by induction on l(w0 J trivially true. So suppose that l(w0 ) l(u) > 0 and let s D(u) and us W J . We have four cases to consider. a) s D(v ) and vs W J . Then the result follows immediately by induction. b) s D(v ) and us vs. We know that [u, v ]J is a chain, so us vs implies that us = v . It follows that

Eu,v (q ) = (q 1)Eus,v (q ) = q (1 q 1 ). c) s D(v ) and us vs. We have Eu,v (q ) = (q q 1 )q l(v)l(u)1 (1 q 1 ) + q l(v)l(u)2 (1 q 1 ) = q l(v)l(u)2 (1 q 1 )q 2 = q l(v)l(u) (1 q 1 ).

180

RICCARDO BIAGIOLI

Figure 9. d) vs W J . Then the result follows immediately by induction. By the comment preceding the proposition this settles the case (Bn , Bn1 ). Let us examine the case (Dn , Dn1 ). Using the notations of 5 we let W = Dn , S = {s0 , . . . , sn1 }, and s0 = (1, 2)(1, 2). Now we let J := S \ {sn1 }. The quotient can be written in the form W J = {w W : w1 (2) < w1 (1) < . . . < w1 (n 1)}, and its Bruhat order is drawn in Figure 9. Moreover we know the unique reduced expression of each w W J , in fact the n 1 elements in the chain on the bottom are e, sn1 , . . . , sn1 s2 , the n 1 elements in the chain on the top are sn1 s2 s0 s1 , . . . , sn1 s2 s0 s1 s2 sn1 , and the remaining two elements are exactly sn1 s2 s0 and sn1 s2 s1 . So every i [2, n 1] identies two elements in the quotient, ui := sn1 si in the chain on the bottom, and vi := sn1 s2 s0 s1 si in the chain on the top. Moreover, we dene un := e. Note that for all i [2, n] and for all j [1, n 1] we have lD (ui ) = n i and lD (vj ) = n + j 1. We call (u, v ) a equidistant pair if there exists i [2, n] such that u = ui and v = vi1 . The verication of the following observation is left to the reader. Lemma 6.2. Let u, v W J . If (u, v ) is an equidistant pair, then u1 (j ) = v 1 (j ) for all j [2, n 1]. Proposition 6.3. Let u, v W J . Then Eu,v (q ) = (q l(v)l(u) q )(1 q 1 ) q l(v)l(u) (1 q 1 ) if (u, v ) is equidistant, otherwise.

Proof. We have three cases to consider. 1) (u, v ) = (ui , vj ), with i [2, n] and j [1, n 1] \ {i 1}. If j i, then ui = sn1 si and vj = sn1 s2 s0 s1 sj . Hence si1 D(ui ) and ui si1 = ui1 W J , while vj si1 W J , so we have Eui ,vj (q ) = qEui1 ,vj (q ). We have i 2 steps like this to do, and one more step for s = s0 , so at the end we

CLOSED PRODUCT FORMULAS FOR VERMA MODULES

181

have (37) Eui ,vj (q ) = q i2 Eu2 ,vj (q ) = q i1 Eu2 s0 ,vj (q ). Now [u2 s0 , vj ]J is a chain with j steps and so by Proposition 6.1 it follows that Eu2 s0 ,vj (q ) = q j 1 (q 1) and so from (37) we are done. If j < i 1, after (i j 2) steps in the diagram we have (38) Eui ,vj (q ) = q ij 2 Euj+2 ,vj (q ). The next step is for s = sj +1 ; sj +1 D(uj +2 ) and sj +1 D(vj ), so by the rst part of the proof we have Euj+2 ,vj (q ) = Euj+1 ,vj+1 (q ) = q 2j +1 (1 q 1 ), and the result follows from (38). 2) (u, v ) is an equidistant pair. We proceed by induction on i. If i = 2, then u2 = sn1 s2 and v1 = sn1 s2 s0 s1 . Hence s0 D(u2 ), s0 D(v1 ) and u2 s0 v1 s0 , so Eu2 ,v1 (q ) = (q 1)Eu2 s0 ,v1 (q ). In the second step, s := s1 , we have u2 s0 s1 v1 s1 , so Eu2 ,v1 (q ) = (q 1)2 Eu2 s0 s1 ,v1 (q ) = (q 1)2 , since v1 = u2 s0 s1 . So suppose i > 2. Then si1 D(ui ), ui si1 = ui1 W J , and vi1 si1 = vi2 W J , so since ui1 vi2 , we have Eui ,vi1 (q ) = (q q 1 )Eui1 ,vi1 (q ) + Eui1 ,vi2 (q ). By case 1), Eui1 ,vi1 (q ) = q l(vi1 )l(ui1 ) (1 q 1 ) and by induction, Eui1 ,vi2 (q ) = q l(vi2 )l(ui1 ) (1 q 1 ). The result follows. 3) [u, v ]J is a chain. The result follows by Proposition 6.1. This completes the proof. 7. Consequences and further remarks In this section we derive some consequences of our results. We start by proving that the E -polynomials are combinatorial invariants, i.e. that they depend only on the poset [u, v ]J . To do this we need a purely order theoretic result on skew partitions that was rst proved in [4, Lemma 5.5]. Lemma 7.1. Let , be two connected skew partitions that are isomorphic as posets. Then either or . We can now prove the main result of this section. Corollary 7.2. Let J S , as in 3, 4, and 5, and u, v W J , x, y W J be such that [u, v ]J = [x, y ]J . Then Eu,v (q ) = Ex,y (q ). Proof. Now we prove this result in the case when W = Sn . By Proposition 3.2 we have that [u, v ]J is isomorphic, as a poset, to the interval [(u), (v )] in Youngs lattice. But it follows immediately from the denitions and well-known results in the theory of partially ordered sets (see, e.g., [18, 3.4]) that the subposet of joinirreducibles of [(u), (v )] is isomorphic to (v ) \ (u), where the skew partition (v ) \ (u) is seen as a poset. Therefore, since [u, v ]J = [x, y ]J , we conclude that (v ) \ (u) = (y ) \ (x) (as poset), and the result follows from Lemma 7.1 and Corollary 3.6.

182

RICCARDO BIAGIOLI

Similarly, we can prove the result for the other cases, but we need to replace Proposition 3.2, Corollary 3.6 and for (Cn , An1 ) with Proposition 4.2, Corollary 4.7 and B , and for (Dn , An1 ) with Proposition 5.2, Corollary 5.6 and D , respectively. Note that in the case when W = Sn the proof of Corollary 7.2 applies whenever [u, v ]K = [x, y ]H with K, H S , |K | = |H | = |S | 1. We conclude this section by discussing the connections mentioned at the end of the Introduction. In [14], Gabber and Joseph dene for every u, v W a polynomial Ru,v (q ) =
k 0

(1)l(v)l(u)k q k dim(Ext (Mu , Mv ))

k

and they conjectured (although this is not explicitly stated) that Ru,v (q ) = Ru,v (q ). This conjecture is not true (see [2]), but the R and the R-polynomials are not so dierent. In fact, Carlin shows that the R -polynomials are monic of degree l(v ) l(u) (see [8, Theorem 3.8]), as are the R-polynomials, and proves that the conjecture is true in two cases: when l(v ) l(u) 3 (see [8, Proposition 3.13]) and when (u, v ) is a Coxeter pair ([8, Proposition 3.11]). The E -polynomials play the same role as the R -polynomials in the generalized case, so it is natural to J J (q ), where Ru,v (q ) wonder about the analogous question, i.e. if Eu,v (q ) = Ru,v are the parabolic R-polynomials (see e.g., [12]). This question also has a negative S \{(2,3)} . Then we have that answer. In fact, for example, let v = [3, 4, 1, 2, 5] S5 J 4 1 2 4 Re,v (q ) = q (1 q )(1 q ), while Ee,v (q ) = q (1 q 1 )(1 q 3 ). However, we can prove the analogue of the results of Carlin, for generalized Verma modules. The rst two results are very simple, and their proofs are immediate from Theorems 3.4, 4.5 and 5.5. Corollary 7.3. Let u, v W J , u v . Then Eu,v (q ) is a monic polynomial of degree l(v ) l(u). Corollary 7.4. Let u, v W J . If u v , then dim(Extl(v)l(u) (Nu , Nv )) = 1.

In [4] Brenti nds explicit formulas for the maximal parabolic R-polynomials of the symmetric group and, in [5] for the group of signed permutations, when J = S \ {s0 }. He proves the following
J , u v . Then Theorem 7.5. Let u, v Sn J (q ) = q |(v)\(u)| Ru,v j j ((u),(v )) (1 q a )

where j runs over the allowable steps of (v ) with respect to (u).

B J ) , u v . Then Theorem 7.6. Let u, v (Sn J (q ) = q |B (v)\B (u)| Ru,v j

(1 q bj (B (u),B (v)) )

CLOSED PRODUCT FORMULAS FOR VERMA MODULES

183

where j runs over the B -allowable steps of B (v ) with respect to B (u) and bj (B (u), B (v )) := a j (B (u), B (v )) a j (B (u), B (v )) + 1 if the j -th step of B (u) is down, if the j -th step of B (u) is up.

The author will prove in another paper that the formula in Theorem 7.6 can be D , but we need to replace B by D . extended to the case W = Sn Our formulas are very similar to Brentis, and it is easy to see that the following results hold. The statements and proofs are given for W = Sn , but the results are B D and Sn . We simply need to replace by B and D , respectively. also true for Sn We say that a skew partition is a border strip (also called a ribbon) if it contains no 2 2 square of cells. Proposition 7.7. Let u, v W J be such that the skew partition (v ) \ (u) is a border strip. Then
J (q ). Eu,v (q ) = Ru,v

So, in particular, we obtain the analogue of Proposition 3.13 of [8]. Corollary 7.8. Let u, v W J . If l(v ) l(u) 3, then
J (u, v ), dim(Extk (Nu , Nv )) = rk J J (u, v ) is the absolute value of the coecient of q k in Ru,v (q ). where rk J (u, v ) = l(v ) l(u). Let u, v W J ; we call (u, v ) a generalized Coxeter pair if r1 The next result is the analogue of Proposition 3.11 of [8].

Proposition 7.9. Let u, v W J . If (u, v ) is a generalized Coxeter pair, then

J (u, v ) = dim(Extk (Nu , Nv )) = rk

n , k

for k = 0, . . . , n, where n = l(v ) l(u).

J (q ) Proof. If (u, v ) is a generalized Coxeter pair, then the only possibility is that Ru,v n 1 n = q (1 q ) . This means that the skew partition (v ) \ (u) is a border strip, and so the result follows by Proposition 7.7.

It seems that the situation is exactly analogous, hence we are led to think that, as for the generalized Verma modules, there should also exist a recursion formula for the ordinary ones. So, we close this paper with this proposal. Problem. Find a recursion formula for the R -polynomials. Acknowledgements I am very grateful to my advisor, Francesco Brenti, for suggesting the problem to me and especially for his generous support. I would like to thank David Vogan and George Lusztig for some useful conversations, and M.I.T. for its hospitality during the preparation of this work.

184

RICCARDO BIAGIOLI

References
[1] I.N. Bernstein, I.M. Gelfand, S.I. Gelfand, On a category of g -modules, Funct. Anal. Appl., 10 (1976), 1-8. MR 53:10880 [2] B. Boe, A Counterexample to the Gabber-Joseph Conjecture, Kazhdan-Lusztig Theory and related topics, Contemp. Math., 139, 1-2, Amer. Math. Soc., Providence, RI, 1992. MR 94a:22032 [3] B. Boe, T.J. Enright, B. Shelton, Determination of the interwining operators for holomorphically induced representations of Hermitian symmetric pairs, Pacic J. Math., 131 (1988), 39-50. MR 88m:22038 [4] F. Brenti, Kazhdan-Lusztig and R-polynomials, Youngs lattice, and Dyck partitions, Pacic J. Math., to appear. [5] F. Brenti, personal communication, December 2000. [6] F. Brenti, q-Eulerian Polynomials Arising from Coxeter Groups, Europ. J. Combinatorics, 15 (1994), 417-441. MR 95i:05013 [7] J. Brylinski, M. Kashiwara, Kazhdan-Lusztig conjecture and holonomic systems, Invent. Math., 64 (1981), 387-410. MR 83e:22020 [8] K.J. Carlin, Extensions of Verma Modules, Trans. Amer. Math. Soc., 294 (1986), 29-43. MR 87j:17006 [9] L. Casian, D. Collingwood, The Kazhdan-Lusztig conjecture for generalized Verma modules, Math. Z., 195 (1987), 581-600. MR 88i:17008 [10] D.H. Collingwood, The n-Homology of Harish-Chandra Modules: Generalizing a theorem of Kostant, Math. Ann., 272 (1985), 161-187. MR 87a:22027 [11] D.H. Collingwood, B. Shelton, A duality theorem for extensions on induced Highest Weight modules, Pacic. J. Math., 146 (1990), 227-237. MR 91m:22029 [12] V. Deodhar, On some geometric aspects of Bruhat orderings. II. The parabolic analogue of Kazhdan-Lusztig polynomials, J. Algebra, 111 (1987), 483-506. MR 89a:20054 [13] T.J. Enright, B. Shelton, Categories of highest weight modules: applications to Hermitian symmetric spaces, Mem. Amer. Math. Soc., 367 (1987). MR 88f:22052 [14] O. Gabber, A. Joseph, Towards the Kazdhan-Lusztig conjecture, Ann. Sci. Ecole. Norm. Sup., 14 (1981), 261-302. MR 83e:17009 [15] J. E. Humphreys, Reection Groups and Coxeter Groups, Cambridge Stud. Adv. Math., no. 29, Cambridge Univ. Press, Cambridge, 1990. MR 92h:20002 [16] D. Kazhdan, G. Lusztig, Representations of Coxeter groups and Hecke algebras, Invent. Math., 53 (1979), 165-184. MR 81j:20066 [17] B. Shelton, Extensions Between Generalized Verma Modules: The Hermitian Symmetric Case, Math. Z., 197 (1988), 305-318. MR 89d:17006 [18] R. P. Stanley, Enumerative Combinatorics, vol. 1, Wadsworth and Brooks/Cole, Monterey, CA, 1986. MR 87j:05003 [19] R. P. Stanley, Enumerative Combinatorics, vol. 2, Cambridge Stud. Adv. Math., no. 62, Cambridge Univ. Press, Cambridge, 1999. MR 2000k:05026 ` di Roma La Sapienza, 00185 Roma, Italy Dipartimento di Matematica, Universita Current address : LACIM, Universit e du Queb ec ` a Montr eal, case postale 8888, succursale Centre-Ville, Montr eal, Queb ec, Canada H3C 3P8 E-mail address : [email protected]