'

&

$

%

Department of Mathematics

Recursive Formula forE[ ∏k

i=0 Tr{(WΣ−1)mi}], WhereW ∼ Wp(I, n) in Finite and Asymptotic

Regime

Jolanta Pielaszkiewicz, Dietrich von Rosen and Martin Singull

LiTH-MAT-R--2015/04--SE

Department of MathematicsLinkoping University

S-581 83 Linkoping, Sweden.

Recursive formula for E[∏k

i=0 Tr{(WΣ−1)mi}],

where W ∼ Wp(Σ, n) in finite and asymptotic regime

Jolanta Pielaszkiewicz∗, Dietrich von Rosen�,∗ and Martin Singull∗

∗Department of Mathematics,Linkoping University,

SE–581 83 Linkoping, Sweden.E-mail: jolanta.pielaszkiewicz@liu.se

E-mail: martin.singull@liu.se

� Department of Energy and Technology,Swedish University of Agricultural Sciences,

SE–750 07 Uppsala, Sweden.E-mail: Dietrich.von.Rosen@slu.se

Abstract

In this paper, we give a general recursive formula for E[∏k

i=0 Tr{Wmi}], whereW ∼ Wp(I, n) denotes a real Wishart matrix. Formulas for fixed n, p are presented

as well as asymptotic versions when np

n,p→∞→ c, i.e., when the so called Kolmogorovcondition holds. Finally, we show application of the asymptotic moment relation whenderiving moments for the Marchenko-Pastur distribution (free Poisson law). A numer-ical illustration using implementation of the main result is also performed.

Keywords: Wishart matrix; spectral distribution; eigenvalue distribution; moments;random matrices; free Poisson law; Marchenko-Pastur law.

1 Introduction

Multivariate analysis and random matrix theory are useful tools in mathematics, financialmathematics, statistics, engineering, physics as well as other disciplines too. The matrix dis-tribution which is nowadays known as the Wishart distribution was first derived by Wishart(1928). It is usually regarded as an extension of the chi-square distribution. Wishart ma-trices are, commonly used in statistics. For example, under a multivariate Gaussian model(normality), the distribution of the sample-covariance matrix is Wishart distributed. More-over, many test statistics considered in classical multivariate analysis are given as a functionof one or several Wishart matrices. In this work the expectation of the product of traces of

1

Wishart matrices is studied. These quantities can be used when approximating densities.For example, Kawasaki and Seo (2012) considered tests for mean vectors with unequal co-variance matrices and then used moments of trace products of Wishart matrices. Moreover,when expanding the Stieltjes transform moments of products of traces appear.

LetX ∈ Rp×n follow the central matrix normal distribution, denotedX ∼ Np,n(0,Σ, In),where the dispersion matrix Σ in this work is assumed to be positive definite, denotedΣ > 0, and In is the identity matrix of size n × n. Alternatively, one can think abouta set of n independently distributed p-dimensional column vectors Xi, i = 1, . . . , n, eachdistributed according to a multivariate normal distribution, i.e., Np(0,Σ). Then, W =XX ′ =

∑ni=1XiX

′i, where X = (X1, . . . , Xn) : p × n and X ′ denotes the transpose of X,

follows a central real Wishart distribution, W ∼ Wp(Σ, n). In fact, if there exist a normallydistributed X, with mean 0, and W = XX ′ then W is said to be Wishart distributed.Another way of defining the Wishart distributions is to utilize the Laplace transform whichprovides a somewhat more general definition of the Wishart distribution.

Let for an arbitrary matrix A the matrix Ak denote

k times︷ ︸︸ ︷AA · · ·A where usual matrix multi-

plication is applied. Our main result provides a new recursive, non–combinatorial, formulafor

E[k∏i=0

Tr{Wmi}], mi ≥ 0, i = 0, . . . , k, (1)

where W ∼ Wp(I, n), E[·] denotes expectation and the trace Tr{·} is defined as the sum ofthe diagonal elements of a square matrix. The special case E[Tr{W l}] coincides with thefree moments of the empirical spectral distribution for W . Asymptotically (np → c > 0)delivers the moments of the free Poisson law, see Marchenko and Pastur (1967). Observethat since the free Poisson distribution is compactly supported the moments carry the fullinformation about the distribution. Thus, it is interesting to put energy into the derivationof moment relations which can be interpreted and easily used in computations. Moreover,our expressions can be used to prove recursive formulas for the Catalan numbers. Moreabout Catalan number can be found in Grimaldi (2011).

The new moment formula, given in this paper, is an extension of results provided byFujikoshi, Ulyanov and Shimizu (2011), see Theorem 2.2.6, when W ∼ Wp(I, n). It is alsoan extension of formulas given in Gupta and Nagar (2000). We will compare our resultswith formulas for the complex Wishart matrix presented in Haagerup and Thorbjornsen(2003) and Hanlon, Stanley and Stembridge (1992). The presented result provides also aformula for E[(Tr{W})k] as an alternative to the results given by Letac and Massam (2004),see further Section 2.

Furthermore, a relatively new idea when deriving moment relations is to apply so calledumbra calculations. For example, see Di Nardo (2014) who studied relations similar to aswell as more general than (1).

The paper is organized as follows. In Section 2 some previously obtained results forspecial cases of (1) are recalled, which indeed motivate this research. Section 3 comprisesthe derivation of the main result, i.e., a general recursive formula for (1). A number ofcorollaries will be presented as a consequences of the formula. Moreover, in Section 4

2

a theorem about asymptotic expressions is given. Then, in Section 5, we recall the freePoisson law and use our result to rediscover its moments. A recursive formula for Catalannumbers is also presented. At the end of the paper, in Section 6, a flowchart is given forcomputing moments explicitly together with a few numerical results.

2 A Survey of Known Results

In this section for both real and complex Wishart matrices a number of results that are ofthe same form as (1) will be considered.

2.1 Real Wishart matrix

Let W ∼ Wp(Σ, n). Then, one can easily see that Σ−1/2WΣ−1/2 ∼ Wp(I, n). Thus,E[Tr{Σ−1W}] equals E[Tr{V }] if V ∼ Wp(I, n).

The expectation of the power of the trace of the Wishart matrix, W ∼ Wp(Σ, n),has been derived by Nel (1971) for specific powers k. The general formula involve zonalpolynomials and following Theorem 3.3.23 in Gupta and Nagar (2000) is given by

E[(Tr{W})k] = 2k∑κ

[n

2

]κ

Zκ(Σ),

where Zκ(·) stands for zonal polynomial corresponding to κ, κ = (k1, . . . , km) is a partitionof k, such that ki ≥ 0 for all i = 1, . . . ,m and

∑i ki = k, [a]κ =

∏mi=1[a − i + 1]ki with

[a]k = (a+ k)!/a! for a ∈ C and k ∈ N0. The alternative version of the closed formula forthe expectation of the power of the trace of the Wishart matrix W can be found in Mathai(1980).

Using results for zonal polynomials, see e.g., Subrahmaniam (1976),

E[(Tr{Σ−1W})k] = 2k[np

2

]k

.

For any k ∈ N, a theorem stated in Letac and Massam (2004) gives an alternative formula

E[(Tr{Σ−1W})k] =∑

(i)∈Ik

k!

i1! · · · ik!1i1 · · · kik(np)i1+...+ik2i2+2i3+...+(k−1)ik , (2)

where the set Ik consists of k-tuples (i) = (i1, . . . , ik) such that i1 + 2i2 + . . .+ kik = k andij , j = 1, . . . , k are non-negative integers. Moreover, several other results and referencesare mentioned in Gluckmuller (1998).

Above the expectation of the power of the trace of a real central Wishart matrix hasbeen presented. Other formulations are about the expectation of the trace of the power, i.e.,E[Tr{(Σ−1W )l}]. For a non-central Wishart distribution, V ∼ Wp(Σ, n,Θ), i.e., a Wishartmatrix which is defined via X ∼ Np,n(M,Σ, I) as V = XX ′ with Θ = MM ′, it is wellknown that the following holds (see Gupta and Nagar (2000) for more details)

E[Tr{V Σ−1}] = np+ Tr{Θ},E[Tr{(V Σ−1)2}] = np(n+ p+ 1) + 2(n+ p+ 1)Tr{Θ}+ Tr{Θ2}. (3)

3

Moreover, for W ∼ Wp(Σ, n) (see e.g., Fujikoshi, Ulyanov and Shimizu (2011))

E[Tr{W 2}] = (n+ n2)Tr{Σ2}+ n(Tr{Σ})2. (4)

Letac and Massam (2008) derived an expression for

E[

k∏i=0

Tr{(WHi)}], (5)

where H1, . . . ,Hk are arbitrary real symmetric matrices. Choosing Hi appropriately mo-ments of all monomials are available. Using umbral calculations a generalization of (5) toa non-central Wishart matrix has been suggested by Di Nardo (2014).

Note that the notations in the papers of Letac and Massam (2008) and Di Nardo (2014)differ somewhat from ours since the Wishart distribution, e.g., in Letac and Massam isbased on the Laplace transform with a different Σ.

2.2 Complex Wishart matrix

The complex p-dimensional column vector Xi follows the complex normal distribution Xi ∼NCp (µ,Σ) if the 2p-dimensional vector Yi = (<X ′i,=X ′i)′ ∼ N2p((<µ′,=µ′)′,ΣYi), where

ΣYi =1

2

(<Σ −=Σ=Σ <Σ

),

=Xi (resp. <Xi) denotes the imaginary (resp. real) part of the complex Xi and the matrix=Σ is skew-symmetric.

An early reference to the complex Wishart distribution is Goodman (1963). Let againX = (X1, . . . , Xn) where Xi ∼ NC

p (µ,Σ) which are supposed to be independently dis-

tributed. Then, a matrix WC ∼ WCp (Σ, n) is complex Wishart distributed if WC = XX∗,

where ∗ denotes the conjugate transpose. An alternative definition of the complex Wishartdistribution is based on the Laplace transform.

Let WC ∼ WCp (I, n). The matrix WC is considered in Hanlon, Stanley and Stembridge

(1992), where it was obtained that

E[Tr{W kC}] =

1

k

k∑j=1

(−1)j−1 [n+ k − j]k[p+ k − j]k(k − j)!(j − 1)!

, k ∈ N. (6)

A corresponding recursive result was derived by Haagerup and Thorbjornsen (2003), i.e.,for all k ∈ N the following holds

E[Tr{W 0C}] = p,

E[Tr{W 1C}] = np,

E[Tr{W k+1C }] =

(2k + 1)(n+ p)

k + 2E[Tr{W k

C}]

+(k − 1)(k2 − (n− p)2)

k + 2E[Tr{W k−1

C }]. (7)

4

The given recursive formula (7) was inspired by the Harer-Zagier recursion formula (seeHarer and Zagier (1986)). The original paper Harer and Zagier (1986) gives a recursiverelation for even moments of the spectral distribution of a complex self-adjoint randommatrix Z = (Zij) formed of p2 independent real standard normal distributed variables suchthat Zij has mean 0 and variance 1, i.e., an expression for 1

pE[Tr{Z2k}]. It has been reprovedand extended to the interesting case of Wishart matrices in Haagerup and Thorbjornsen(2003). The proof of (7) is mostly combinatorial while the proof given in Harer and Zagier(1986) combines advanced tools from analysis and theory of solving differential equations.Both the explicit result given in Hanlon, Stanley and Stembridge (1992) and the recursiveresult of Haagerup and Thorbjornsen (2003) derive E[Tr{W k

C}], although equation (7) issaid by Haagerup and Thorbjornsen (2003) to be more efficient than (6) for generation ofmoment tables.

2.3 Comparison of Moments for Complex and Real Wishart Matrices

The formulas regarding complex Wishart matrices WC and real Wishart matrices W differfor fixed n and p. In Table 1, E[Tr{W k

• }] is presented for k ≤ 5, where W• denotes eitherWC ∼ WC

p (I, n) or W ∼ Wp(I, n). There is a significant difference between the real andcomplex cases. In particular, it means that (7) cannot be applied when WC is replaced byW .

Table 1: Comparison of E[Tr{W k}] and E[Tr{W kC}], when k ≤ 5, WC ∼ WC,p(I, n) and

W ∼ Wp(I, n). The formulas have been derived using (7) and our result presented inTheorem 3.1.

k E[Tr{W k}] E[Tr{W kC}]

1 np np2 np(n+ p+ 1) np(n+ p)3 np(n2 + p2 + np+ 3n+ 3p+ 4) np(n2 + p2 + np+ 1)4 np(n3 +p3 + 6n2p+ 6np2 + 6n2 +

6p2 + 17np+ 21n+ 21p+ 20)np(n3+p3+6n2p+6np2+5n+5p)

5 np(p4 + n4 + 10np3 + 10n3p +20n2p2 + 61n2 + 61p2 + 163np +10n3 + 10p3 + 55n2p + 55np2 +148n+ 148p+ 132)

np(n4 + p4 + 10n3p + 10np3 +20n2p2 + 15n2 + 15p2 + 40np+ 8)

Using (7)

E[Tr{W 2C}] =

3(n+ p)

3E[Tr(WC)] = np(n+ p),

which differs from the real case, i.e., E[Tr{W 2}] = np(n + p + 1). We can also see thedifference through a simple example.

Example 2.1. Let X be 2 × 2 real matrix such that Xij ∼ N(0, 1) and let the matrix W

5

be given as

W = XX ′ =

(X2

11 +X212 X11X21 +X12X22

X11X21 +X12X22 X222 +X2

21

).

Hence, since W ∼ W2(I2, 2),

E[TrW ] = E[X211 +X2

12 +X222 +X2

21] = 4,

E[Tr{W 2}] = E[(X211 +X2

12)2 + 2(X11X21 +X12X22)2 + (X222 +X2

21)2] = 20,

which is consistent with the formulas (3) and (4). Similarly, let us consider the case with

a 2 × 2 complex matrix X such that Xjk =Yjk+iZjk√

2, Yjk, Zjk ∼ N(0, 1) and Yjk, Zjk are

independent for all j, k = 1, 2. Then, Xjk has mean 0 and variance 1 and the matrix

WC = XX∗ ∼ WC2 (I2, 2). Furthermore,

E[TrWC] =1

2E[Y 2

11 + Z211 + Y 2

12 + Z212 + Y 2

21 + Z221 + Y 2

22 + Z222] = 4,

E[Tr{W 2C}] =

1

4E[(Y 2

11 + Z211 + Y 2

12 + Z212)2 + 2(Y12Y22 + Z12Z22 + Y11Y21 + Z11Z21)2

+2(Z12Y22 − Y12Z22 + Z11Y21 − Y11Z21)2 + (Y 221 + Z2

21 + Y 222 + Z2

22)2]

= 16.

The difference between the results for the real and complex Wishart matrices of fixedsize increases together with the power k. It can be observed that it is given by a polynomialof degree lower than k + 1.

3 A New Recursive Moment Formula

In this paper, the main goal is to generalize all the results for a real Wishart matrix whichwere presented in the previous by proposing a recursive formula for E[

∏ki=0 Tr{(WΣ−1)mi}],

where W ∼ Wp(Σ, n) for any k ∈ N and mi ∈ N0, i = 1, . . . , k. Essential to the proof ofthe forthcoming Theorem 3.1 is the use of the operator d

dX , which can be considered asdifferentiation with respect to a symmetric matrix X. For Y ∈ Rq×r and X ∈ Rp×p

dY

dX=∑I

∂yij∂xkl

(gl ⊗ gk)εkl(ej ⊗ di)′, εkl =

{1 : k = l,12 : k 6= l,

(8)

where I = {i, j, k, l : 1 ≤ i ≤ q, 1 ≤ j ≤ r, 1+ ≤ k ≤ p, 1 ≤ l ≤ n}, di, ej and gk are i-th,j-th and k-th column of Iq, Ir and Ip, respectively, and ⊗ denotes the Kronecker product.Rules of how to operate with (8) are given in A.

Moreover, the Wishart density function representingWp(Σ, n) is used in the proof. Eventhough the density exists only for n ≥ p, the result given below holds as well in the casep ≥ n, due to rotational invariance, i.e., Tr{Y Y ′} = Tr{Y ′Y }.

6

Theorem 3.1. Let W ∼ Wp(I, n). Then, the following recursive formula holds for allk ∈ N and all m0, m1, . . . ,mk such that m0 = 0, mk ∈ N, mi ∈ N0, i = 1, . . . , k − 1

E[ k∏i=0

Tr{Wmi}]

= (n− p+mk − 1)E[Tr{Wmk−1}

k−1∏i=0

Tr{Wmi}]

+2

k−1∑i=0

miE[Tr{Wmk+mi−1}

k−1∏j=0j 6=i

Tr{Wmj}]

+

mk−1∑i=0

E[Tr{W i}Tr{Wmk−1−i}

k−1∏j=0

Tr{Wmj}]. (9)

Remark 3.1. Note, that the presented formula is recursive with respect to the power mk.

Proof. Put mk = l + 1. Then,

Σ−1E[ k−1∏i=0

Tr{(WΣ−1)mi}(WΣ−1)l]

=∫

Σ−1∏k−1i=0 Tr{(WΣ−1)mi}(WΣ−1)lfWdW, (10)

where fW denotes the density function for W , i.e., if Σ > 0 and n ≥ p

fW (W ) =1

2pn2 Γp(

n2 )|Σ|−

n2 |W |

n−p−12 e−

12Tr{Σ−1W}, W > 0.

Note that as Tr{WΣ−1} is rotationaly invariant the result will also hold for p ≥ n.On equation (10) we operate with d

dΣ−1 and then apply the trace Tr{·}. Note that LHS(RHS) stands for left hand side (right hand side) of (10).Then, since E

[∏k−1i=0 Tr{(WΣ−1)mi}(WΣ−1)l

]does not depend on Σ

d(LHS)

dΣ−1=

dΣ−1

dΣ−1

(E[ k−1∏i=0

Tr{(WΣ−1)mi}(WΣ−1)l]⊗ Ip

)

=1

2(Ip2 +Kp,p)

(E[ k−1∏i=0

Tr{(WΣ−1)mi}(WΣ−1)l]⊗ Ip

),

where Kp,p stands for the commutation matrix (see Appendix B or Kollo and von Rosen(2005)). Moreover,

Tr

{d(LHS)

dΣ−1

}= Tr

{1

2(Ip2 +Kp,p)

(E[ k−1∏i=0

Tr{(WΣ−1)mi}(WΣ−1)l]⊗ Ip

)}

=1

2E[ k−1∏i=0

Tr{(WΣ−1)mi}Tr{(WΣ−1)l}Tr{Ip}]

+1

2E[ k−1∏i=0

Tr{(WΣ−1)mi}Tr{(WΣ−1)lIp}]

7

=p+ 1

2E[ k−1∏i=0

Tr{(WΣ−1)mi}Tr{(WΣ−1)l}].

For the RHS the following holds

d(RHS)

dΣ−1=

∫dfWdΣ−1

vec′(Σ−1

k−1∏i=0

Tr{(WΣ−1)mi}(WΣ−1)l)dW︸ ︷︷ ︸

=B

+

∫d(∏k−1i=0 Tr{(WΣ−1)mi}Σ−1(WΣ−1)l)

dΣ−1fWdW︸ ︷︷ ︸

=C

.

SincedfWdΣ−1

=1

2(nvecΣ− vecW )fW ,

where (19) and (21) has been applied

B = E[

1

2(nvecΣ− vecW )vec′

(Σ−1

k−1∏i=0

Tr{(WΣ−1)mi}(WΣ−1)l)]

and thus

Tr{B} =n

2E[ k−1∏i=0

Tr{(WΣ−1)mi}Tr{(WΣ−1)l}]− 1

2E[ k−1∏i=0

Tr{(WΣ−1)mi}Tr{(WΣ−1)l+1}].

Using the product rule (19) we have that C = C1 + C2, where

C1 = E[d(Σ−1(WΣ−1)l)

dΣ−1

k−1∏i=0

Tr{(WΣ−1)mi}]

= E[d(W−1(WΣ−1)l+1)

dΣ−1

k−1∏i=0

Tr{(WΣ−1)mi}]

(15)= E

[d((WΣ−1)l+1)

dΣ−1(I ⊗W−1)

k−1∏i=0

Tr{(WΣ−1)mi}]

(20)= E

[d(WΣ−1)

dΣ−1

{ ∑i+j=l

(WΣ−1)i ⊗ (WΣ−1)′j}

(I ⊗W−1)

k−1∏i=0

Tr{(WΣ−1)mi}]

(15)= E

[dΣ−1

dΣ−1(I ⊗W )

{ ∑i+j=li,j≥0

(WΣ−1)i ⊗ (WΣ−1)′j}

(I ⊗W−1)

k−1∏i=0

Tr{(WΣ−1)mi}]

(12)= E

[Ip2 +Kp,p

2

{ ∑i+j=li,j≥0

(WΣ−1)i ⊗W (WΣ−1)′jW−1

} k−1∏i=0

Tr{(WΣ−1)mi}]

8

=1

2E[{ ∑

i+j=li,j≥0

(WΣ−1)i ⊗W (WΣ−1)′jW−1

} k−1∏i=0

Tr{(WΣ−1)mi}]

+1

2E[Kp,p

{ ∑i+j=li,j≥0

(WΣ−1)i ⊗W (WΣ−1)′jW−1

} k−1∏i=0

Tr{(WΣ−1)mi}]

and

C2 = E[d(∏k−1i=0 Tr{(WΣ−1)mi})

dΣ−1vec′(Σ−1(WΣ−1)l)

].

Hence,

Tr{C1} =1

2

∑i+j=li,j≥0

E[Tr{(WΣ−1)i}Tr{W (WΣ−1)′jW−1}

k−1∏i=0

Tr{(WΣ−1)mi}]

+1

2

∑i+j=li,j≥0

E[Tr{

(WΣ−1)iW (WΣ−1)′jW−1} k−1∏i=0

Tr{(WΣ−1)mi}]

=1

2

∑i+j=li,j≥0

E[Tr{(WΣ−1)i}Tr{(WΣ−1)′j}

k−1∏i=0

Tr{(WΣ−1)mi}]

+1

2

∑i+j=li,j≥0

E[Tr{(WΣ−1)l}

k−1∏i=0

Tr{(WΣ−1)mi}]

=1

2

∑i+j=li,j≥0

E[Tr{(WΣ−1)i}Tr{(WΣ−1)′j}

k−1∏i=0

Tr{(WΣ−1)mi}]

+l + 1

2E[Tr{(WΣ−1)l}

k−1∏i=0

Tr{(WΣ−1)mi}].

Using again the product rule (19) we have

C2 = E[d(∏k−2i=0 Tr{(WΣ−1)mi})

dΣ−1Tr{(WΣ−1)mk−1}vec′(Σ−1(WΣ−1)l)

]︸ ︷︷ ︸

=D1

+E[d(Tr{(WΣ−1)mk−1})

dΣ−1

k−2∏i=0

Tr{(WΣ−1)mi}vec′(Σ−1(WΣ−1)l)

]︸ ︷︷ ︸

=D2

.

9

The integral D2 is computed using the chain rule (18) ddΣ−1 = dWΣ−1

dΣ−1d

dWΣ−1 , and then (12),(16) and (20), which gives

D2 = E[d(Tr{(WΣ−1)mk−1})

dΣ−1

k−2∏i=0

Tr{(WΣ−1)mi}vec′(Σ−1(WΣ−1)l)

](18)= E

[d((WΣ−1)mk−1)

dΣ−1

d(Tr{(WΣ−1)mk−1})d((WΣ−1)mk−1)

k−2∏i=0

Tr{(WΣ−1)mi}vec′(Σ−1(WΣ−1)l)

](18)= E

[d(WΣ−1)

dΣ−1

d((WΣ−1)mk−1)

d(WΣ−1)

d(Tr{(WΣ−1)mk−1})d((WΣ−1)mk−1)

k−2∏i=0

Tr{(WΣ−1)mi}

vec′(Σ−1(WΣ−1)l)

](16)(20)= E

[dΣ−1

dΣ−1(I ⊗W )

{ ∑i+j=mk−1−1

i,j≥0

(WΣ−1)i ⊗ (WΣ−1)′j}

vecI

·k−2∏i=0

Tr{(WΣ−1)mi}vec′(Σ−1(WΣ−1)l)

](12)= E

[1

2(Ip2 +Kp,p)

{ ∑i+j=mk−1−1

i,j≥0

(WΣ−1)i ⊗W (WΣ−1)′j}

vecI

·k−2∏i=0

Tr{(WΣ−1)mi}vec′(Σ−1(WΣ−1)l)

].

Then,

Tr{D2} =1

2

∑i+j=mk−1−1

i,j≥0

E[Tr

{Σ−1(WΣ−1)l(WΣ−1)iI(WΣ−1)jW

k−2∏i=0

Tr{(WΣ−1)mi}}]

+1

2

∑i+j=mk−1−1

i,j≥0

E[Tr

{Σ−1(WΣ−1)lW (WΣ−1)′jI(WΣ−1)′i

k−2∏i=0

Tr{(WΣ−1)mi}}]

= 21

2mk−1E

[Tr{(WΣ−1)l+mk−1}

k−2∏i=0

Tr{(WΣ−1)mi}]

= mk−1E[Tr{(WΣ−1)l+mk−1}

k−2∏i=0

Tr{(WΣ−1)mi}].

10

The integral D1 is of similar form as C2, hence repeating the same calculations we obtain

Tr{C2} =k−1∑i=0

miE[Tr{(WΣ−1)l+mi}

k−1∏j=0j 6=i

Tr{(WΣ−1)mj}].

Finally, we have

p+ 1

2E[ k−1∏i=0

Tr{(WΣ−1)mi}Tr{(WΣ−1)l}]

=n

2E[ k−1∏i=0

Tr{(WΣ−1)mi}Tr{(WΣ−1)l}]

−1

2E[ k−1∏i=0

Tr{(WΣ−1)mi}Tr{(WΣ−1)l+1}]

+1

2

∑i+j=li,j≥0

E[Tr{(WΣ−1)i}Tr{(WΣ−1)′j}

k−1∏j=0

Tr{(WΣ−1)mj}]

+l + 1

2E[Tr{(WΣ−1)l}

k−1∏i=0

Tr{(WΣ−1)mi}]

+k−1∑i=0

miE[Tr{(WΣ−1)l+mi}

k−1∏j=0j 6=i

Tr{(WΣ−1)mj}],

which is equivalent to the statement of the theorem.

If in Theorem 3.1 k = 1 and m1 = l + 1 then the next corollary is obtained.

Corollary 3.1. Let W ∼ Wp(I, n), then for all l ∈ N0

E[Tr{W l+1}] = p[n− p− 1]l+1 +l∑

i=0

[n− p+ i]l−i

i∑j=0

E[Tr{W j}Tr{W i−j}].

The expectation E[Tr{W k}Tr{Wm}

]in Corollary 3.1 can be determined by for example

choosing k = 2, m1 = m and m2 = v in Theorem 3.1.

Corollary 3.2. Let W ∼ Wp(I, n). Then, for all v ∈ N and m ∈ N0,

E[Tr{W v}Tr{Wm}

]= p[n− p− 1]vE[Tr{Wm}]

+2m

v−1∑i=0

[n− p+ i]v−1−iE[Tr{Wm+i}]

+

v−1∑i=0

[n− p+ i]v−1−i

i∑j=0

E[Tr{W j}Tr{W i−j}Tr{Wm}].

11

In order to obtain an expression for E[Tr{W l+1}] it was shown that E[Tr{W j}Tr{W i−j}]was needed where i ≤ l as well as j ≤ l. Thus, we gain in power, l instead of l + 1 butinstead we got the new problem of finding the expectation of the product of two terms. InCorollary 3.2 the product was considered. Once again we gain in power but from now onexpectation of a product of three terms has to be derived. This process can be continuedand at the end the following result is needed:

Corollary 3.3. Let W ∼ Wp(I, n), then the following recursive formula holds for all t ∈ N

E[(Tr{W})t+1

]=(np+ 2t

)E[(Tr{W})t

]=

(np+ 2t)!!

(np)!!np.

Proof. The statement of Corollary 3.3 is a special case of Theorem 3.1, when mi = 1 forall i = 1, . . . , k applied t+ 1 times for k = t+ 1, . . . , 1.

The chain of moment relations initiated in Corollary 3.1 and ended in Corollary 3.3 showsthat any moment of the form E[Tr{W l}] can be expressed explicitly.

4 Asymptotics of 1pk+1ns

E[∏k

i=0 Tr{Wmi}]

Let W ∼ Wp(I, n). Then, E[W l] = O(nl) which will serve as a basis for the forthcomingtheorem. As the considered expectation is increasing together with n and p we are interestedto understand the asymptotic behavior of its normalized version 1

pk+1nsE[∏k

i=0 Tr{Wmi}],

where s =∑k

i=0mi and when np→c while n, p→∞. This means that p and n increase with

the same speed and the criterion is sometimes called the Kolmogorov condition (asymp-totic).

Theorem 4.1. Let W ∼ Wp(I, n), m = {m0, . . . ,mk−1}, lim np = c > 0 and

(k)Q(mk,m) := limn,p→∞

1

pk+1nsE[ k∏i=0

Tr{Wmi}].

Then, for all k ∈ N and all mk, m such that m0 = 0, mk ∈ N, mi ∈ N0 for i = 1, . . . , k− 1and s =

∑ki=0mi the following holds

(k)Q(mk,m) =

(1c + 1)(k)Q(mk − 1,m)

+1c

∑mk−2i=1 (k+1)Q(i, {m0, . . . ,mk−1,mk − 1− i}), mk > 2, k ≥ 1,

(1c + 1)(k)Q(1,m), mk = 2, k ≥ 1,

(k−1)Q(mk−1, {m0, . . . ,mk−2}), mk = 1, k > 1,

1, mk = 1, k = 1.

12

Proof. Let us first notice that the LHS of (9) is a polynomial in n and p of degree k+ 1 + s,similarly to the first and third summand on the RHS. The second summand on the RHS isof lower degree. Hence, when p, n→∞, n

p → c,

1

pk+1n∑k

i=0mi

k−1∑i=0

miE[Tr{Wmk−1+mi}

k−1∏j=0j 6=i

Tr{Wmj}]→ 0.

Therefore,

(k)Q(mk,m)

=

(1− 1

c

)lim

n,p→∞

1

pk+1n∑k

i=0mi−1E[Tr{Wmk−1}

k−1∏i=0

Tr{Wmi}]

+1

clim

n,p→∞

∑i+j=mk−1

i,j≥0

1

pk+2ni+j+∑k−1

i=0 mi

E[Tr{W i}Tr{W j}

k−1∏i=0

Tr{Wmi}]

︸ ︷︷ ︸=:(k+1)Q(i,{m0,...,mk−1,j})

=

(1c + 1)(k)Q(mk − 1,m)

+1c

∑mk−2i=1 (k+1)Q(i, {m0, . . . ,mk−1,mk − 1− i}), mk > 2, k ≥ 1,

(1c + 1)(k)Q(1,m), mk = 2, k ≥ 1,

(k−1)Q(mk−1, {m0, . . . ,mk−2}), mk = 1, k > 1,

1, mk = 1, k = 1.

These limiting results allow us to analyze the asymptotics of the normalized expectation1

pk+1nsE[∏k

i=0 Tr{Wmi}].

5 Moments of the Free Poisson Distribution

A formula for the asymptotic empirical spectral distribution of M = 1nΣ−1/2WΣ−1/2, where

W ∼ Wp(Σ, n), has been given in the late 60’s by Marchenko and Pastur (1967). Sincethen it has been of constant use due to the development of Random Matrix theory and dueto the fact that it arises in Free Probability theory established in Voiculescu (1985, 1991).Frequently the asymptotic empirical spectral distribution of M is called the Marchenko–Pastur law or alternatively, as an analog of the Poisson distribution, the free Poisson law.The support of the distribution is compact and its moments (free moments) are definedas 1

pE[Tr{Mk}], k = 1, 2, . . .. Moreover, the free cumulants can be derived by one of thefree cumulant-moment relations, see classical results in Nica and Speicher (2006), and arecursive alternative in Pielaszkiewicz, von Rosen and Singull (2014).

Note that the formula for the free moments of M , while p, n → ∞, np → c, satisfy the

asymptotic equation derived in Section 4, i.e., 1pE[Tr{M l}]→ (1)Q(l, {∅}), where ∅ denotes

13

the empty set. Thus,

limp,n→∞

1

pE[TrM ] = 1,

limp,n→∞

1

pE[Tr{M2}] = 1 +

1

c,

limp,n→∞

1

pE[Tr{M l+1}] =

(1 +

1

c

)1

plim

p,n→∞E[Tr{M l}] (11)

+1

c

l−1∑i=1

limp,n→∞

1

p2E[Tr{M i}Tr{M l−i}].

Moreover, the free moments can be presented non-recursively:

limp,n→∞

1

pE[TrM ] = 1,

limp,n→∞

1

pE[Tr{M2}] = 1 +

1

c,

limp,n→∞

1

pE[Tr{M3}] =

(1 +

1

c

)2

+1

c= 1 +

3

c+

1

c2,

limp,n→∞

1

pE[Tr{M4}] =

(1 +

1

c

)(1 +

3

c+

1

c2

)+ 2

1

c

(1 +

1

c

)= 1 +

6

c+

6

c2+

1

c3.

These relations are the first four free moments of the free Poisson law and the results arein agreement with Oravecz and Petz (1997), where it was stated that

limp,n→∞

1

pE[Tr{Mk}] =

1

k

k∑i=1

(k

i

)(k

i− 1

)ci.

In the case of c = 1 the moments of the Marchenko-Pastur distribution are given by theCatalan numbers Ck := 1

k+1

(2kk

). Catalan numbers have various applications, for example

in graph theory as a number of all rooted binary trees with k nodes or number of all waysto triangulate a regular k+ 2 sided polygon. Hence, using (11) or computing directly fromthe asymptotic result from Section 4 we have that

Ck+1 = 2Ck +k−1∑i=1

CiCk−i =k∑i=0

CiCk−i,

which agrees with a recursive result for Catalan numbers. Interesting properties and ap-plications of Catalan numbers, along with results for Fibonacci numbers, can be found inGrimaldi (2011).

6 Implementation of results

The recursive formula given in Theorem 3.1 is a convenient tool for calculating non-asymptotic free moments of random Wishart matrices of fixed size, as well as more general

14

moment expressions such as E[∏ki=0 Tr{Wmi}] and its normalized version

1

pk+1n∑k

i=0mi

E[k∏i=0

Tr{Wmi}],

where W ∼ Wp(I, n). In Table 2 a few numerical results are presented for m1 = 1, m2 =4, m3 = 3.

Table 2: Numerical results for 1p3n8E[Tr{W 1}Tr{W 4}Tr{W 3}] with given n and p such that

c = np = 2 and its asymptotics, when W ∼ Wp(I, n).

n 12 24 48 96 120 200E[Tr{W 1}Tr{W 4}Tr{W 3}]

p3n8 41.03 21.99 17.71 16.38 16.17 15.86

n 400 800 2 · 103 2 · 104 2 · 105 2 · 106

E[Tr{W 1}Tr{W 4}Tr{W 3}]p3n8 15.65 15.56 15.50 15.47 15.4691 15.46878

n →∞E[Tr{W 1}Tr{W 4}Tr{W 3}]

p3n8c5+9c4+25c3+25c2+9c+1

c5= 15.46875

The asymptotic values are obtained by utilizing Theorem 4.1 in the following way:

(3)Q(3, {1, 4}) =

(1 +

1

c

)(3)Q(2, {1, 4}) +

1

c(4)Q(1, {1, 4, 1})

=

(1 +

1

c

)2

(3)Q(1, {1, 4}) +1

c(3)Q(1, {1, 4})

=

(1 +

1

c

)2

(2)Q(4, {1}) +1

c(2)Q(4, {1}) =

(1 +

3

c+

1

c2

)(2)Q(4, {1})

=

(1 +

3

c+

1

c2

)((1 +

1

c

)(2)Q(3, {1}) + 2

1

c(3)Q(1, {1, 2})

)=

(1 +

4

c+

4

c2+

1

c3

)((1 +

1

c

)(2)Q(2, {1}) +

1

c(3)Q(1, {1, 1})

)+

(2

c+

6

c2+

2

c3

)(1 +

1

c

)(2)Q(1, {1}) = 1 +

9

c+

25

c2+

25

c3+

9

c4+

1

c5

c=2= 15.46875.

Table 3 presents the moments 1pnmE[Tr{W 4}], where W ∼ Wp(I, n). The asymptotic value

is obtained using Theorem 4.1:

limn,p→∞

1

pn4E[Tr{W 4} = 1 +

6

c+

6

c2+

1

c3

e.g.=

45, c = 0.514, c = 12.448, c = 5

The case c = 1 corresponds to the Marchenko-Pastur distribution.

15

Table 3: Numerical results for 4th moment 1pn4E[Tr{W 4}] for given finite n and p, such

that c = np ∈ {

12 , 1, 5}, and its asymptotic value.

n 6 60 100 400 103 104 →∞c = 1

2 57.51 46.08 45.65 45.16 45.06 45.01 451pn4E[Tr{W 4}], c = 1 20.09 14.50 14.29 14.07 14.03 14.00 14

c = 5 4.847 2.616 2.547 2.472 2.458 2.449 2.448

The values in Table 2 and 3 have been obtained using a function denoted Etr(m,n, p,Q =1), which was implemented in R according to the algorithm given by flowchart in Figure 1.The blue states and dashed blue lines on flowchart indicate the recursive steps, diamondsstand for the decision states and the red parallelograms are the ending states. Yellowblocks indicate parts of algorithm responsible for calculating summands of formula (3.1).The two last summands are calculated using procedure given in the blocks on the right handside of Figure 1. While the first summand as well as final summing of obtained results isimplemented according to block on the left hand side of flowchart.

The algorithm depends on the following four input parameters: a) m - a k-dimensionalvector of powers m1, . . . ,mk which was used in Theorem 3.1; b) n and p - degrees of freedomand size of the Wishart matrix, respectively; c) binary Q which describes if we are interestedin the normalized expectation (Q = 1) or E

[∏ki=0 Tr{(WΣ−1)mi}

](Q = 0).

Example 6.1. The function Etr((4), 120, 24, 1), counting numerical value of the fourthmoment 1

pn4E[Tr{(WΣ−1)4}] with n = 120, p = 24, is executed in R by

> Etr(c(4),120,24,1)

[1] 2.530095

The forth moment E[Tr{(WΣ−1)4}] with n = 120, p = 24, can be obtained in R usingfunction Etr((4), 120, 24, 0)

> Etr(c(4),120,24,0)

[1] 12591371520

Value of 1p3n10E[Tr{(WΣ−1)4}Tr{WΣ−1}Tr{(WΣ−1)5}] with n = 120, p = 24, can be

obtained by

> Etr(c(4,1,5),120,24,0)

[1] 10.6138

The code can be downloaded from http://www.mai.liu.se/~jolpi20/Etr.R.

16

Begin

m,n, p,Q k = length(m)

mk > 0 mk = mk − 1

k > 1

m = (m1, . . . ,mk−1)

temp = p · run(m, n, p,Q)

temp = 0

temp p

mk > 1

i = 1

m = (m1, . . . ,mk−1, i,mk − i)

tempq4[i] = run(m, n, p,Q)

i = i+ 1

i ≤ k − 1

temp4 = sum(tempq4)

k > 1 i = 1

m = (m1, . . . ,mi +mk, . . . ,mk−1)

tempq3[i] = 2mirun(m, n, p,Q)

i = i+ 1

i ≤ k − 1

temp3 = sum(tempq3)mk = 0temp = n · run(m, n, p,Q)

temp = (n+ p+mk)run(m, n, p,Q)

temp+ temp3 + temp4

YES

NO

YES

NO

NO YES

YES

YES

NO

NO

YES

YES

NO

NO

YES

NO

E[Tr{(WΣ−1)0}∏

i Tr{(WΣ−1)mi}]

∑mk−1i=1 E[Tr{(WΣ−1)i}Tr{(WΣ−1)mk−i}

∏j Tr{(WΣ−1)mj }]

2∑

imiE[Tr{(WΣ−1)mi+mk}∏

j Tr{(WΣ−1)mj }]

(n+ p+mk)E[Tr{(WΣ−1)mk}∏

i Tr{(WΣ−1)mi}]or

nE[Tr{(WΣ−1)0}∏

i Tr{(WΣ−1)mi}]

Figure 1: Flowchart of implemented algorithm to calculate E[∏ki=0 Tr{(WΣ−1)mi}].

17

References

E. Di Nardo, On a symbolic representation of non-central Wishart random matriceswith applications, J. Multivar. Anal. 125 (2014) 121–135.

Y. Fujikoshi, V. V. Ulyanov and R. Shimizu, Multivariate Statistics: High-Dimensionaland Large-Sample Approximations (John Wiley & Sons, Hoboken, 2011).

D. H. Glueck and K. E. Muller, On the trace of a wishart, Comunnications in Statistics- Theory and Methods 27 (1998) 2137-2141.

N. R. Goodman, Statistical analysis based on a certain multivariate complex Gaussiandistribution (an introduction), Ann. Math. Statist. 34 (1963) 152–177.

R. Grimaldi, Fibonacci and Catalan Numbers : An Introduction (John Wiley & Sons,Somerset, 2011).

A. K. Gupta and D. K. Nagar, Matrix Variate Distributions, Monographs and surveysin pure and applied mathematics; 104 (Chapman & Hall/CRC, Boca Raton, 2000).

U. Haagerup and S. Thorbjornsen, Random matrices with complex Gaussian entries,Expo. Math. 21 (2003) 293–337.

P. J. Hanlon, R. P. Stanley and J.R. Stembridge, Some combinatorial aspects of thespectral of normally distributed random matrices, Contemp. Math. 138 (1992) 151–174.

J. L. Harer and D. Zagier, The Euler characteristic of the moduli space of curves, Invent.Math. 85(3) (1986) 457–485.

H. Hotelling, A generalized T test and measure of multivariate dispersion, Proc. SecondBerkeley Symp. Math. Statist. Prob. (Univ. of California Press, Berkeley, 1951), 23–42.

T. Kawasaki and T. Seo, A two sample test for mean vectors with unequal covariancematrices, Technical report 12-19, Hiroshima University (2012).

T. Kollo and D. von Rosen, Advanced Multivariate Statistics with Matrices (Springer,Dordrecht, 2005).

M. Ledoux, A recursion formula for the moments of the Gaussian orthogonal ensemble,Ann. Inst. H. PoincarA c© Probab. Statist. 45 No. 3 (2009) 754–769.

G. Letac and H. Massam, All invariant moments of the Wishart distribution, Scand. J.Stat 31 No. 2 (2004) 295–318.

G. Letac and H. Massam, The noncentral Wishart as an exponential family, and itsmoments, J. Multivar. Anal. 99 No. 1 (2008) 1393–1417.

V. A. Marchenko and L. A. Pastur, Distribution of eigenvalues in certain sets of randommatrices, Mat. Sb. (N.S.) 72(114):4 (1967) 507–536.

A. M. Mathai, Moments of the trace of a noncentral wishart matrix, Commun. Stat.Theory Methods 9:8 (1980) 795–801.

18

D. G. Nel, The h-th moment of the trace of a noncentral Wishart matrix, S. Afr. Statist.J. 5 (1971) 41–52.

A. Nica and R. Speicher, Lectures on the Combinatorics of Free Probability (CambridgeUniversity Press, Cambridge, 2006).

F. Oravecz and D. Petz, On the eigenvalue distribution of some symmetric randommatrices, Acta Sci. Math. 63 (1997) 383–395.

J. Pielaszkiewicz, D. von Rosen and M. Singull, Cumulant-moment relation in freeprobability theory, to appear in Acta Comment. Univ. Tartu. Math. 18(2) (2014).

R. Speicher, Multiplicative functions on the lattice of noncrossing partitions and freeconvolution, Math. Ann. 298 (1994) 611–628.

K. Subrahmaniam, Recent trends in multivariate normal distribution theory: On thezonal polynomials and other functions of matrix argument, Sankhya Ser. A 38 No. 3(1976) 221–258.

D. Voiculescu, Symmetries of some reduced free product C∗–algebras, Operator algebrasand their connections with topology and ergodic theory, in Proc. Conf., Busteni/Rom.,Lect. Notes Math. 1132 (1985) pp. 556–588.

D. Voiculescu, Limit laws for Random matrices and free products, Invent. math. 104(1991) 201–220.

J. Wishart, The generalized product moment distribution in samples from a normalmultivariate population, Biometrika 20A (1928) 32–52.

19

A Properties of Operator ddX

The operator defined in (8) has the following properties:

dX

dX=

1

2(I +Kp,p) for symmetric X, (12)

d(aY )

dX= a

dY

dX, (13)

d(Z + Y )

dX=

dZ

dX+dY

dX, (14)

d(AXB)

dX= B ⊗A′, (15)

d(AY B)

dX=

dY

dX(B ⊗A′), (16)

dTr{A′X}dX

= vecA, (17)

dZ

dX=

dY

dX

dZ

dY, (18)

dW

dX=

dY

dX

dW

dY+dZ

dX

dW

dZ,where W = W (Y (X), Z(X)), (19)

dY n

dX=

dY

dX

( ∑i+j=n−1i,j≥0

Y i ⊗ (Y ′)j), (20)

d|X|dX

= |X|vec(X−1)′. (21)

Proof. Eq. (12)

dX

dX=

∑I

∂xij∂xkl

(fl ⊗ gk)εkl(fj ⊗ gi)′

X=X′=

∑k,lk 6=l

∂xkl∂xkl

(fl ⊗ gk)1

2(fl ⊗ gk)′ +

∑k,lk 6=l

∂xlk∂xkl

(fl ⊗ gk)1

2(fk ⊗ gl)′

+∑k

∂xkk∂xkk

(fk ⊗ gk)(fk ⊗ gk)′

=1

2

∑k,lk 6=l

(fl ⊗ gk)[(fl ⊗ gk)′ + (fk ⊗ gl)′] +∑k

(fk ⊗ gk)(fk ⊗ gk)′

=1

2

∑k,l

(fl ⊗ gk)[(fl ⊗ gk)′ + (fk ⊗ gl)′] =1

2(I +Kp,p)

20

Eq. (13)

d(aY )

dX=

∑I

∂(ayij)

∂xkl(fl ⊗ gk)εkl(ej ⊗ di)′

= a∑I

∂yij∂xkl

(fl ⊗ gk)εkl(ej ⊗ di)′ = adY

dX

Eq. (14)

d(Z + Y )

dX=

∑I

∂(zij + yij)

∂xkl(fl ⊗ gk)εkl(ej ⊗ di)′

=∑I

∂zij∂xkl

(fl ⊗ gk)εkl(ej ⊗ di)′ +∑I

∂yij∂xkl

(fl ⊗ gk)εkl(ej ⊗ di)′

=dZ

dX+dY

dX

Eq. (15). Equivalently dYdX can be given as

dY

dX=dvec′Y

dvecX,

then

d(AXB)

dX=

d

=vec′X(B⊗A′)︷ ︸︸ ︷vec′(AXB)

dvecX=dvec′X(B ⊗A′)

dvecX=dvec′X

dvecX(B ⊗A′)

=dX

dX(B ⊗A′) = B ⊗A′

Eq. (16)

d(AY B)

dX=

dY

dX

d(AY B)

dY=dY

dX(B ⊗A′)

Eq. (17)

dTr{A′X}dX

(18)Tr{Y }=vec′IvecY

=d(A′X)

dX

d(vec′Ivec(A′X))

d(A′X)

=d(A′X)

dX

d(vec′(A′Xvec′I))

d(A′X)

=d(A′X)

dX

d(A′X)

d(A′X)vecI

(15)= (I ⊗A)vecI = vecA

21

Eq. (18)

dZ

dX=

∑I

∂zij∂xkl

(fl ⊗ gk)εkl(ej ⊗ di)′

=∑i,j,k,l

∑m,n

∂zij∂ymn

∂ymn∂xkl

(fl ⊗ gk)εkl(hn ⊗ km)′(hn ⊗ km)εnm(ej ⊗ di)′

=∑

i,j,k,l,m,n,o,p

∂zij∂ymn

∂yop∂xkl

(fl ⊗ gk)εkl(hp ⊗ ko)′(hn ⊗ km)εnm(ej ⊗ di)′

=∑k,l,o,p

∂yop∂xkl

(fl ⊗ gk)εkl(hp ⊗ ko)′∑i,j,m,n

∂zij∂ymn

(hn ⊗ km)εnm(ej ⊗ di)′

=dY

dX

dZ

dY

Eq. (19). As

∂wij∂xgh

=∑m,n

∂ymn∂xgh

∂wij∂ymn

+∑m,n

∂zmn∂xgh

∂wij∂zmn

then by the same argument as in proof of chain rule 6. the statement is proven.

Eq. (20) (mathematical induction). For n=1 by Eq. 5. dYdX = dY

dX (I ⊗ I).For n = 2 by Eq. 7.

dY 2

dX=dY

dX(Y ⊗ I) +

dY

dX(I ⊗ Y ′) =

dY

dX

( ∑i+j=1i,j≥0

Y i ⊗ (Y ′)j).

Let assume statement is true for n = k − 1. Then

dY k

dX=

d(Y Y k−1)

dX

prop.7.=

dY k−1

dX(I ⊗ Y ′) +

dY

dX(Y k−1 ⊗ I)

ind.assump.=

dY

dX

( ∑i+j=k−2i,j≥0

Y i ⊗ (Y ′)j)(I ⊗ Y ′) +

dY

dX(Y k−1 ⊗ I)

=dY

dX

( ∑i+j=k−2i,j≥0

Y i ⊗ (Y ′)j+1)

+dY

dX(Y k−1 ⊗ I)

=dY

dX

( ∑i+j=k−1i,j≥0

Y i ⊗ (Y ′)j)

what finishes the proof.

22

Eq. (21). Determinant |X| and element of inverse matrix X−1 can be written as

|X| =∑j

xij(−1)i+j |X(ij)|,

(X−1)ij =(−1)i+1|X(ji)|

|X|,

where X(ij) denotes the minor of an element xij . Then as for all indeces i, j

∂|X|∂xij

= (−1)i+j |X(ij)| = (X−1)ij |X|

the statement of (21) holds.

B Commutation matrix Kp,p

The commutation matrix is defined as

Kp,p =∑I

eie′j ⊗ eje′i,

where I = {i, j, k, l : 1 ≤ i ≤ p, 1 ≤ j ≤ p} and ei is i-th column of Iq. Let vec(•) denotethe usual vec-operator. Then, assuming sizes are appropriate,

Tr{AB} = vec′(A′)vec(B), (22)

Tr{A⊗B} = Tr{A}Tr{B}, (23)

Tr{Kp,p(A⊗B)} = Tr{AB}. (24)

23