and, in addition, use the relation

y C ‚x,y f(x, y) = ‚x,y (xC f(x, y)) ’ (‚x,y xC )f(x, y) (3.7)

repeatedly.

A “q-Abel-type” variation of this result reads as follows.

ADVANCED DETERMINANT CALCULUS 29

Theorem 24. Let n be a nonnegative integer, and let Bm (X) denote the n — m matrix

«

[C]2 [C]m’1

1 [C]q ...

q q

¬X ·

2

[C + 1]m’1 X

[C + 1]q X [C + 1]q X ...

¬2 ·

q

¬X [C + 2]m’1 X 2 · ,

[C + 2]q X 2 [C + 2]2 X 2 ...

¬ ·

q q

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

X n’1 [C + n ’ 1]q X n’1 [C + n ’ 1]2 X n’1 . . . [C + n ’ 1]m’1 X n’1 q q

i.e., any next column is formed by applying the operator X 1’C Dq X C , with Dq denoting

the q-derivative as in Theorem 23. Given a composition of n, n = m1 + · · · + m , there

holds

det Bm1 (X1 ) Bm2 (X2 ) . . . Bm (X )

1¤i,j,¤n

mi ’1 mj ’1

mi ’1

(mi )

(q t’s Xj ’ Xi ), (3.8)

= q N2 2

Xi [j]q !

i=1 j=1 s=0 t=0

1¤i<j¤

where N2 is the quantity

mi

((C + j + m1 + · · · + mi’1 ’ 1)(mi ’ j)) ’ ’ mj

mj mi

mi .

2 2

i=1 j=1 1¤i<j¤

Yet another generalization of the Vandermonde determinant evaluation is found in

[21]. Multidimensional analogues are contained in [176, Theorem A.7, Eq. (A.14),

Theorem B.8, Eq. (B.11)] and [182, Part I, p. 547].

Extensions of Cauchy™s double alternant (2.7) can also be found in the literature (see

e.g. [117, 149]). I want to mention here particularly Borchardt™s variation [17] in which

the (i, j)-entry in Cauchy™s double alternant is replaced by its square,

’ Xj )(Yi ’ Yj )

1¤i<j¤n (Xi

1 1

det = Per , (3.9)

(Xi ’ Yj )2 1¤i,j¤n (Xi ’ Yj )

1¤i,j¤n Xi ’ Yj

1¤i,j¤n

where Per M denotes the permanent of the matrix M. Thus, there is no closed form

expression such as in (2.7). This may not look that useful. However, most remarkably,

there is a (q-)deformation of this identity which did indeed lead to a “closed form evalu-

ation,” thus solving a famous enumeration problem in an unexpected way, the problem

of enumerating alternating sign matrices.10 This q-deformation is equivalent to Izergin™s

evaluation [74, Eq. (5)] (building on results by Korepin [82]) of the partition function of

the six-vertex model under certain boundary conditions (see also [97, Theorem 8] and

[83, Ch. VII, (10.1)/(10.2)]).

An alternating sign matrix is a square matrix with entries 0, 1, ’1, with all row and column

10

sums equal to 1, and such that, on disregarding the 0s, in each row and column the 1s and (’1)s

alternate. Alternating sign matrix are currently the most fascinating, and most mysterious, objects in

enumerative combinatorics. The reader is referred to [18, 19, 111, 148, 97, 198, 199] for more detailed

material. Incidentally, the “birth” of alternating sign matrices came through ” determinants, see

[150].

30 C. KRATTENTHALER

Theorem 25. For any nonnegative integer n there holds

’ Xj )(Yi ’ Yj )

1¤i<j¤n (Xi

1

det =

(Xi ’ Yj )(qXi ’ Yj ) 1¤i,j¤n (Xi ’ Yj )(qXi ’ Yj )

1¤i,j¤n

n

N i (A)

Ni (A)

— (1 ’ q) (±i,j Xi ’ Yj ), (3.10)

2N (A)

Xi Yi

i=1

A i,j such that Aij =0

where the sum is over all n — n alternating sign matrices A = (Aij )1¤i,j¤n , N(A) is

the number of (’1)s in A, Ni (A) (respectively N i (A)) is the number of (’1)s in the

i-th row (respectively column) of A, and ±ij = q if j Aik = i Akj , and ±ij = 1

k=1 k=1

otherwise.

Clearly, equation (3.9) results immediately from (3.10) by setting q = 1. Roughly,

Kuperberg™s solution [97] of the enumeration of alternating sign matrices consisted of

suitably specializing the xi ™s, yi ™s and q in (3.10), so that each summand on the right-

hand side would reduce to the same quantity, and, thus, the sum would basically count

n — n alternating sign matrices, and in evaluating the left-hand side determinant for

that special choice of the xi ™s, yi ™s and q. The resulting number of n — n alternating

sign matrices is given in (A.1) in the Appendix. (The ¬rst, very di¬erent, solution

is due to Zeilberger [198].) Subsequently, Zeilberger [199] improved on Kuperberg™s

approach and succeeded in proving the re¬ned alternating sign matrix conjecture from

[111, Conj. 2]. For a di¬erent expansion of the determinant of Izergin, in terms of Schur

functions, and a variation, see [101, Theorem q, Theorem γ].

Next we turn to typical applications of Lemma 3. They are listed in the following

theorem.

Theorem 26. Let n be a nonnegative integer, and let L1, L2 , . . . , Ln and A, B be inde-

terminates. Then there hold

1¤i<j¤n [Li ’ Lj ]q

n

i=1 [Li + A + 1]q !

Li + A + j n

(i’1)(Li +i)

det = q i=1 ,

[A + 1 ’ i]q !

n n

Li + j [Li + n]q !

1¤i,j¤n

q i=1 i=1

(3.11)

and

1¤i<j¤n [Li ’ Lj ]q i=1 [A + i ’ 1]q !

n

A n

jLi iLi

det q =q , (3.12)

i=1

[A ’ Li ’ 1]q !

n n

Li + j i=1 [Li + n]q !

1¤i,j¤n

q i=1

and

BLi + A

det

Li + j

1¤i,j¤n

1¤i<j¤n (Li ’ Lj )

n n

(BLi + A)!

(A ’ Bi + 1)i’1 , (3.13)

=

((B ’ 1)Li + A ’ 1)!

n

i=1 (Li + n)! i=1 i=1

and

n

(A + BLi )j’1 (A + Bi)i’1

(Lj ’ Li ).

det = (3.14)

(j ’ Li )! (n ’ Li )!

1¤i,j¤n

i=1 1¤i<j¤n

ADVANCED DETERMINANT CALCULUS 31

(For derivations of (3.11) and (3.12) using Lemma 3 see the proofs of Theorems 6.5

and 6.6 in [85]. For a derivation of (3.13) using Lemma 3 see the proof of Theorem 5

in [86].)

Actually, the evaluations (3.11) and (3.12) are equivalent. This is seen by observing

that

’A ’ 1

Li + A + j Li j

= (’1)Li +j q ( 2 )+(2)+jLi +(A+1)(Li +j) .

Li + j Li + j q

q

Hence, replacement of A by ’A ’ 1 in (3.11) leads to (3.12) after little manipulation.

The determinant evaluations (3.11) and (3.12), and special cases thereof, are redis-

covered and reproved in the literature over and over. (This phenomenon will probably

persist.) To the best of my knowledge, the evaluation (3.11) appeared in print explicitly

for the ¬rst time in [22], although it was (implicitly) known earlier to people in group

representation theory, as it also results from the principal specialization (i.e., set xi = q i ,

i = 1, 2, . . . , N) of a Schur function of arbitrary shape, by comparing the Jacobi“Trudi

identity with the bideterminantal form (Weyl character formula) of the Schur function

(cf. [105, Ch. I, (3.4), Ex. 3 in Sec. 2, Ex. 1 in Sec. 3]; the determinants arising in the

bideterminantal form are Vandermonde determinants and therefore easily evaluated).

The main applications of (3.11)“(3.13) are in the enumeration of tableaux, plane par-

titions and rhombus tilings. For example, the hook-content formula [163, Theorem 15.3]

for tableaux of a given shape with bounded entries follows immediately from the the-

ory of nonintersecting lattice paths (cf. [57, Cor. 2] and [169, Theorem 1.2]) and the

determinant evaluation (3.11) (see [57, Theorem 14] and [85, proof of Theorem 6.5]).

MacMahon™s “box formula” [106, Sec. 429; proof in Sec. 494] for the generating function

of plane partitions which are contained inside a given box follows from nonintersecting

lattice paths and the determinant evaluation (3.12) (see [57, Theorem 15] and [85, proof

of Theorem 6.6]). The q = 1 special case of the determinant which is relevant here is

the one in (1.2) (which is the one which was evaluated as an illustration in Section 2.2).

To the best of my knowledge, the evaluation (3.13) is due to Proctor [133] who used

it for enumerating plane partitions of staircase shape (see also [86]). The determinant

evaluation (3.14) can be used to give closed form expressions in the enumeration of »-

parking functions (an extension of the notion of k-parking functions such as in [167]), if

one starts with determinantal expressions due to Gessel (private communication). Fur-

ther applications of (3.11), in the domain of multiple (basic) hypergeometric series, are

found in [63]. Applications of these determinant evaluations in statistics are contained

in [66] and [168].

It was pointed out in [34] that plane partitions in a given box are in bijection with

rhombus tilings of a “semiregular” hexagon. Therefore, the determinant (1.2) counts

as well rhombus tilings in a hexagon with side lengths a, b, n, a, b, n. In this regard,

generalizations of the evaluation of this determinant, and of a special case of (3.13),

appear in [25] and [27]. The theme of these papers is to enumerate rhombus tilings of

a hexagon with triangular holes.

The next theorem provides a typical application of Lemma 4. For a derivation of this

determinant evaluation using this lemma see [87, proofs of Theorems 8 and 9].

32 C. KRATTENTHALER

Theorem 27. Let n be a nonnegative integer, and let L1 , L2 , . . . , Ln and A be indeter-

minates. Then there holds

Li + A ’ j

q jLi

det

Li + j

1¤i,j¤n

q

n

[Li + A ’ n]q !

n

[Li ’ Lj ]q [Li + Lj + A + 1]q . (3.15)

iLi

=q i=1

[Li + n]q ! [A ’ 2i]q ! 1¤i<j¤n

i=1

This result was used to compute generating functions for shifted plane partitions of

trapezoidal shape (see [87, Theorems 8 and 9], [134, Prop. 4.1] and [135, Theorem 1]).

Now we turn to typical applications of Lemma 5, given in Theorems 28“31 below.

All of them can be derived in just the same way as we evaluated the determinant (1.2)

in Section 2.2 (the only di¬erence being that Lemma 5 is invoked instead of Lemma 3).

The ¬rst application is the evaluation of a determinant whose entries are a product

of two q-binomial coe¬cients.

Theorem 28. Let n be a nonnegative integer, and let L1, L2 , . . . , Ln and A, B be inde-

terminates. Then there holds

Li + A ’ j

Li + j

·

det

B B

1¤i,j¤n

q q

( )+2(n+1)

n

i=1 (i’1)Li ’B 2

n

[Li ’ Lj ]q [Li + Lj + A ’ B + 1]q

=q 3

1¤i<j¤n

n

[Li + 1]q ! [Li + A ’ n]q ! [A ’ 2i ’ 1]q !

— . (3.16)

[Li ’ B + n]q ! [Li + A ’ B ’ 1]q ! [A ’ i ’ n ’ 1]q ! [B + i ’ n]q ! [B]q !

i=1

As is not di¬cult to verify, this determinant evaluation contains (3.11), (3.12), as

well as (3.15) as special, respectively limiting cases.

This determinant evaluation found applications in basic hypergeometric functions

theory. In [191, Sec. 3], Wilson used a special case to construct biorthogonal rational

functions. On the other hand, Schlosser applied it in [157] to ¬nd several new summation

theorems for multidimensional basic hypergeometric series.

In fact, as Joris Van der Jeugt pointed out to me, there is a generalization of Theo-

rem 28 of the following form (which can be also proved by means of Lemma 5).

ADVANCED DETERMINANT CALCULUS 33

Theorem 29. Let n be a nonnegative integer, and let X0 , X1 , . . . , Xn’1 , Y0 , Y1 , . . . ,

Yn’1 , A and B be indeterminates. Then there holds

«

Yj + A ’ Xi

Xi + Yj

¬ ·

j j

¬ q·

q

det ¬ ·

0¤i,j¤n’1 Xi + B A + B ’ Xi

j j

q q

n n’1

i(Xi +Yi ’A’2B)

= q 2(3 )+ [Xi ’ Xj ]q [Xi + Xj ’ A]q

i=0

0¤i<j¤n’1

n’1

(q B’Yi ’i+1 )i (q Yi +A+B+2’2i )i

— . (3.17)

(q Xi’A’B )n’1 (q Xi+B’n+2 )n’1

i=0

As another application of Lemma 5 we list two evaluations of determinants (see below)

where the entries are, up to some powers of q, a di¬erence of two q-binomial coe¬cients.

A proof of the ¬rst evaluation which uses Lemma 5 can be found in [88, proof of

Theorem 7], a proof of the second evaluation using Lemma 5 can be found in [155,

Ch. VI, §3]. Once more, the second evaluation was always (implicitly) known to people

in group representation theory, as it also results from a principal specialization (set

xi = q i’1/2, i = 1, 2, . . . ) of a symplectic character of arbitrary shape, by comparing the

symplectic dual Jacobi“Trudi identity with the bideterminantal form (Weyl character

formula) of the symplectic character (cf. [52, Cor. 24.24 and (24.18)]; the determinants

arising in the bideterminantal form are easily evaluated by means of (2.4)).

Theorem 30. Let n be a nonnegative integer, and let L1 , L2 , . . . , Ln and A be indeter-

minates. Then there hold

A A

q j(Lj ’Li ) ’ q j(2Li+A’1)

det

j ’ Li ’j ’ Li + 1

1¤i,j¤n

q q

n

[A + 2i ’ 2]q !

[Lj ’ Li ]q [Li + Lj + A ’ 1]q (3.18)

=

[n ’ Li ]q ! [A + n ’ 1 + Li ]q ! 1¤i<j¤n

i=1 1¤i¤j¤n

and

A A

q j(Lj ’Li ) ’ q j(2Li+A)

det

j ’ Li ’j ’ Li

1¤i,j¤n

q q

n

[A + 2i ’ 1]q !

[Lj ’ Li ]q

= [Li + Lj + A]q . (3.19)

[n ’ Li ]q ! [A + n + Li ]q ! 1¤i<j¤n

i=1 1¤i¤j¤n

A special case of (3.19) was the second determinant evaluation which Andrews needed

in [4, (1.4)] in order to prove the MacMahon Conjecture (since then, ex-Conjecture)

about the q-enumeration of symmetric plane partitions. Of course, Andrews™ evaluation

proceeded by LU-factorization, while Schlosser [155, Ch. VI, §3] simpli¬ed Andrews™

proof signi¬cantly by making use of Lemma 5. The determinant evaluation (3.18)

34 C. KRATTENTHALER

was used in [88] in the proof of re¬nements of the MacMahon (ex-)Conjecture and the

Bender“Knuth (ex-)Conjecture. (The latter makes an assertion about the generating

function for tableaux with bounded entries and a bounded number of columns. The

¬rst proof is due to Gordon [59], the ¬rst published proof [3] is due to Andrews.)

Next, in the theorem below, we list two very similar determinant evaluations. This

time, the entries of the determinants are, up to some powers of q, a sum of two q-

binomial coe¬cients. A proof of the ¬rst evaluation which uses Lemma 5 can be found

in [155, Ch. VI, §3]. A proof of the second evaluation can be established analogously.

Again, the second evaluation was always (implicitly) known to people in group represen-

tation theory, as it also results from a principal specialization (set xi = q i, i = 1, 2, . . . )

of an odd orthogonal character of arbitrary shape, by comparing the orthogonal dual

Jacobi“Trudi identity with the bideterminantal form (Weyl character formula) of the

orthogonal character (cf. [52, Cor. 24.35 and (24.28)]; the determinants arising in the

bideterminantal form are easily evaluated by means of (2.3)).

Theorem 31. Let n be a nonnegative integer, and let L1 , L2 , . . . , Ln and A be indeter-

minates. Then there hold

A A

q (j’1/2)(Lj ’Li ) + q (j’1/2)(2Li+A’1)

det

j ’ Li ’j ’ Li + 1

1¤i,j¤n

q q

n

[A + 2i ’ 1]q !

(1 + q Li +A/2’1/2)

=

(1 + q i+A/2’1/2) [n ’ Li ]q ! [A + n + Li ’ 1]q !

i=1

— [Lj ’ Li ]q [Li + Lj + A ’ 1]q (3.20)

1¤i<j¤n

and

A A

q (j’1/2)(Lj ’Li ) + q (j’1/2)(2Li+A’2)

det

j ’ Li ’j ’ Li + 2

1¤i,j¤n

q q

n

[A + 2i ’ 2]q !