INTRODUCTION
Let x_{ij} = (x_{ij1}, ..., x_{ijp})’, j = 1,
..., n_{i}, i = 1,2, be random samples drawn from independent multivariate
normal populations N_{p}(μ_{i}, Σ_{i}), where
all the parameters are unknown. It is a requirement in many statistical techniques,
such as in discriminant analysis, testing the equality of two mean vectors,
testing the equality of two mean subvectors, to know whether covariance matrices
of the two populations are equal or not (Johnson, 1998;
Krzanowski, 2000; Srivastava, 2002;
Gamage and Mathew, 2008; Fujikoshi
et al., 2010). Before applying any further analysis, this equality
must be tested. The widely used traditional technique for testing the hypothesis
that H_{0}: Σ_{1} = Σ_{2} = Σ against
H_{1}: Σ_{1}≠Σ_{2} where Σ is the
common unknown covariance matrix of the two populations, when the sample sizes
n_{i} larger than the number of variables, p, is the modified likelihood
ratio test. However, in applications concerning modern sciences and economics,
the data consist of very large number of variables taken from small samples.
For instance, DNA microarrays typically measure thousands to millions of gene
expressions on the small sample sizes (Dudoit et al.,
2002; Ibrahim et al., 2002; Sebastiani
et al., 2006; Huang et al., 2009).
When the data have p≥n_{i}, called highdimensional data, the sample
covariance matrices S_{i} are singular making the modified likelihood
ratio test is not valid. The tests under this problem were recently worked by
Schott (2007), Srivastava (2007)
and Srivastava and Yanagihara (2010). To have more powerful
choice of test statistic for testing H_{0} against H_{1} when
p≥n_{i}, a new test statistic is proposed. It is shown that this
proposed test statistic is asymptotically distributed as the standard normal
distribution for any type of common covariance matrix considered with large
p, n_{i}.
Let n_{i} be the sample size drawn from population i, i = 1, 2 and
n = n_{1}+n_{2}2, the following assumptions are made:
where, a_{k} = (trΣ^{k})/p and a_{ji} = (trΣ^{j}_{i})/p.
Let:
and
Since S_{1} and S_{2} are independent estimates of the covariance
matrices Σ_{1} and Σ_{2}, respectively, with (n_{i}1)
S_{i} ∼W_{p} (Σ_{i}, n_{i}1), i =
1, 2, where W_{p} (Σ_{i}, n_{i}1) is a Wishart
distribution with n_{i}1 degree of freedom and covariance matrix Σ_{i}
then the common covariance matrix Σ can be estimated by:
Let:
and
The modified likelihood ratio test suggested by Bartlett
(1937) on an intuitive ground is based on the statistic:
and is valid when p<n_{i}. In particularly, if p is fixed, the asymptotic
null distribution of 2 log L, as n_{i}→∞, for i = 1, 2, is chisquared
distribution with p(p+1)/2 degree of freedom.
Because of the unavailability of the modified likelihood ratio test L when
p≥n_{i}, Schott (2007) proposed a test for
the equality of several covariance matrices. This study then considers Schott’s
test statistic only for the case of two covariance matrices. Based on the consistent
estimator of the square of Frobenius norm of Σ_{1}Σ_{2},
namely tr(Σ_{1}Σ_{2})^{2} his test statistic
is given by:
Under the null hypothesis, T_{J} is asymptotically distributed as N
(0, 1) as (p, n_{1}, n_{2})→∞.
Srivastava (2007) proposed a test based on a lower
bound on Frobenius norm. It is given by:
where:
and
where, c_{0} = n (n^{3}+6n^{2}+21n+18), c_{1}
= 2n (2n^{2}+6n+9), c_{2} = 2n (3n+2) and c_{3} = n
(2n^{2}+5n+7). Under the null hypothesis T_{S} is asymptotically
distributed as N (0, 1) as (p, n)→∞.
Srivastava and Yanagihara (2010) proposed an alternative
test based on a consistent estimator of a measure of distance by ,
where ,
i = 1, 2. The consistent estimators of γ_{i} are given by .
The test statistic is given by:
where:
and
Under the null hypothesis, T_{SY} is asymptotically distributed as
N (0, 1) as (p, n)→∞.
THE PROPOSED STATISTIC
To test the hypothesis H_{0}: Σ_{1} = Σ_{2}
= Σ against H_{1}: Σ_{1}≠Σ_{2} for
p≥n_{i} it is observed that if Σ_{1} = Σ_{2},
then .
Thus under the null hypothesis, the measurement .
Using lemma A3 extended from lemma A1 obtained from Srivastava
(2005) in the Appendix, a consistent estimator of b can be estimated by
.
The following lemma gives the asymptotic distribution of the consistent estimators.
Lemma 1: Let (n_{i}1) S_{i}∼W_{p} (Σ_{i},
n_{i}1), â_{2i}, i = 1, 2, as defined in (1) and ,
then under the assumptions (A2) and (A4):
where,
denotes x converges in distribution to y.
Proof: Since random samples x_{ij}, j = 1, ..., n_{i},
i = 1,2 are drawn from two independent populations and sample covariance matrices
S_{i} are calculated from corresponding independent random samples x_{1j}
and x_{2j} thus, S_{1} and S_{2} must be independent
of each other. In fact, the statistic â_{21} is a function of
S_{1} alone while the statistic â_{22} is also a function
of S_{2} alone. Thus â_{21 }and â_{22} are
also independent and then it makes COV (â_{21}, â_{22})
= 0. By lemma A4 in the Appendix, â_{2i}, i = 1, 2 are asymptotically
normally distributed with mean a_{2i} and variance:
and the fact that the covariance between â_{21} and â_{22}
is zero, it follows that the jointly asymptotic distribution of statistics â_{21}
and â_{22} are the bivariate normal distribution with mean vector
and covariance matrix as given above. The proof is completed.
Note that
is a ratio of two uncorrelated estimators. By the delta method (Lehmann
and Romano, 2005), it ensures that a function of two random variables can
be approximated as normal distribution. The following theorem establishes the
asymptotic normality of the statistic .
Theorem 1: Let b and
be as defined above. Then, under the assumptions (A1)(A4), ,
where:
Proof: We note that .
Hence the partial derivative of
with respect to â_{21} is:
Similarly, the partial derivative of
with respect to â_{22} is:
Thus, by applying the delta method,
asymptotically with:
The proof is completed.
Corollary 1: Let
be as defined above. Under H_{0}: Σ_{1} = Σ_{2}
= Σ and the assumptions (A1)(A4), then:
Proof: Under H_{0}, then a_{21} = a_{22} = a_{2}
and a_{41} = a_{42} = a_{4}. Thus:
It follows Theorem 1, then the proof is completed.
In order to use T in practice, we have to estimate δ^{2} involving
estimate of a_{2} and a_{4}. Under the null hypothesis and by
using consistent estimators of a_{2} and a_{4} as â_{2}
and â^{*}_{4} as given in lemmas A1 and A5 in the Appendix,
respectively and by assumption (A2), we obtained a corresponding consistent
estimator of δ^{2} namely
as:
Thus a test of H_{0} is based on the statistic:
and also its asymptotic null distribution is the standard normal. The proposed
test statistic T* with α level of significance rejects H_{0} if
T*>z_{α/2} where z_{α/2} denotes the upper α/2
quantile of the standard normal distribution.
SIMULATION STUDY
Here, the performance of the proposed test statistic T* compared to three tests
T_{J}, T_{S} and T_{SY} was shown through numerical
simulation technique. In order to assess being normality of the tests, the Attained
Significance Level (ASL) of these tests were simulated and expected to be close
to the nominal significance level setting. The empirical powers of these tests
in different situations were also performed.
Parameter selection: Independent 10,000 replications of the multivariate
normal random datasets were generated using International Mathematics and Statistics
Library (IMSL) with multivariate normal random number generator (RNMVN) subroutine
of Fortran programming language (FORTRAN). The nominal significance level α
used was 0.05. Under the null hypothesis, the test statistics T*, T_{J},
T_{S} and T_{SY} were computed and the proportions of rejection
of test statistics under the null hypothesis were recorded, called the Attained
Significance Level (ASL). In our work presented here, the ASL under the null
hypothesis and corresponding empirical power under the alternative hypothesis
were manipulated for following hypotheses in different patterns of covariance
matrix setup as follow:
• 
Unstructured pattern (UN): It is defined as Σ
= (σ_{ij})^{p}_{i,j=1}. We considered the hypothesis
as follows: 

• 
:
Σ_{1} = Σ_{2} = U_{0} against 

• 
:
Σ_{1} = U_{0} and Σ_{2} = U_{1} 

where, U_{0} = (σ_{ij})^{p}_{i,j=1}
where σ_{ij} = 1 (if i = j); σ_{ij} = (1) ^{i+j}
(0.10i)/j (if i≠ j) and U_{1} = (σ_{ij})^{p}_{i,
j=1} where σ_{ij} = 1 (if i = j); σ_{ij}
= (1)^{i+j} (0.05i)/j (if i ≠ j) 
• 
Compound Symmetry pattern (CS): It
is defined as Σ = σ^{2}I_{p}+k1_{p}1'_{p},
where σ^{2}>0, k is appropriate constant, I_{p}
denotes the pxp identity matrix and 1_{p} denotes the px1 vector
of ones. The hypothesis was set as: 

• 
:
Σ_{1} = Σ_{2} = C_{0} = 0.99I_{p}+(0.01)1_{p}1'_{p}
against 

• 
:
Σ_{1} = C_{0} and Σ_{2} = C_{1}
= 0.95I_{p}+(0.05)1_{p}1'_{p} 
• 
Heterogeneous compound symmetry pattern (CSH): It is
defined as Σ = (σ_{ij})^{p}_{i,j = 1}
where σ_{ij} = σ^{2}_{i} >0 (if i =
j); σ_{ij} = σ_{i}σ_{j}ρ (if
i ≠ j), where ρ is the correlation parameter satisfying ρ<1.
The hypothesis was set as: 

• 
:
Σ_{1} = Σ_{2} = M_{0} where M_{0}
is matrix in CSH with σ_{ij}∼U (5,6) (if i = j), ρ
= 0.5, against 

• 
:
Σ_{1} = M_{0} and Σ_{2} = M_{1}
where M_{1} is matrix in CSH with σ_{ij} ∼U (4,5)
(if i = j), ρ = 0.4. 
• 
Simple pattern (SIM): It is defined as Σ = σ^{2}I
We set the hypothesis testing according to: 

• 
:
Σ_{1} = Σ_{2} = 2I against :
Σ_{1} = 2I and Σ_{2} = 1.5I 

• 
:
Σ_{1} = Σ_{2} = Σ = I_{p} against
:
Σ_{1} = I_{p} and Σ_{2} = Diag (1,1,1,2,
..., 1,1,1,2) 
RESULTS AND DISCUSSIONS
Table 1 presents the ASL and empirical powers of T_{J},
T_{S}, T_{SY} and T* when all covariance matrices in the hypothesis
were under unstructured pattern (UN). The ASL of the tests T_{S} and
T_{SY} were not close to the nominal significance level 0.05 and much
lower than it for all cases considered. The test T_{J} generally yielded
the ASL not close to 0.05 for all cases considered. Moreover, the test T_{J}
gave the ASL around 0.060 when the sample sizes were small, n_{1} =
n_{2} = 20 here and tended to increase when the sample sizes became
larger for any p. For instance, when p = 80, the ASL of T_{J} was 0.057
(at n_{1} = n_{2} = 20) and increased to 0.061 (at n_{1}
= n_{2} = 80). From this table, it is observed that the ASL of the proposed
test T* were reasonably close to 0.05 and get better when p and the sample sizes
increased. This is clear that the tests T_{J}, T_{S} and T_{SY}
were not reasonable tests whereas the proposed test T* were. Considering the
power of the test, since the competitive tests T_{J}, T_{S}
and T_{SY} were not reasonable tests at this situation, then their empirical
powers provided in Table 1 will be skipped. As shown from
this table, the empirical powers of the proposed test T* increased to one when
p and the sample sizes increased. In addition, the empirical powers of the proposed
test T* increased for increasing the sample sizes when p is fixed. For instance,
when p = 160, the empirical power of the proposed test T* was 0.288 (at n_{1}
= n_{2} = 20) and increased to 0.944 (at n_{1} = n_{2}
= 160).
Table 1: 
ASL of T_{J}, T_{S}, T_{SY} and T*
under
and their empirical powers under
and
applied at α = 0.05 

ASL: The attained significance level, U_{0} and U_{1}:
The matrices defined under unstructured pattern (UN), α: The nominal
significance level 
Table 2: 
ASL of T_{J}, T_{S}, T_{SY} and T*
under
and their empirical powers under
and
applied at α = 0.05 

ASL: The attained significance level, C_{0} and C_{1}:
The matrices defined under compound symmetry pattern (CS), α: The nominal
significance level 
Table 2 reports the ASL and empirical powers of T_{J},
T_{S}, T_{SY} and T* when all covariance matrices in the hypothesis
were set under compound symmetry pattern (CS). Both tests T* and T_{J}
gave the satisfactory ASL which quite controlled 0.05 for all cases considered
whereas those of T_{S} and T_{SY} were not close to 0.05. As
seen in the table, the ASL of T_{S} and T_{SY} decreased as
p and the sample sizes increased. For instance, when p = n_{1} = n_{2}
= 80, the ASL of T_{S} and T_{SY} were 0.049 and 0.045, respectively
and both decreased to 0.043 and 0.031 when p = n_{1} = n_{2}
= 160, respectively. Moreover, when p is fixed, the ASL of T_{S} and
T_{SY} were dropped when the sample sizes increased. For example, when
p = 160 ASL of T_{S} and T_{SY} were 0.059 and 0.057 (at n_{1}
= n_{2} = 20), respectively and both values decreased to 0.043 and 0.031,
respectively (at n_{1} = n_{2} = 160). This indicates that the
tests T_{S} and T_{SY} were not suitable whereas the proposed
test T* and T_{J} test were appropriate. This table reports that the
empirical powers of the proposed test T* and T_{J} test were quite high
and rapidly tended to one. Moreover, the empirical powers of both tests were
quite responsive to the increase of p and the sample sizes. Furthermore, the
empirical powers of the proposed test T* were slightly higher than those of
the test T_{J} in cases considered.
Table 3 displays the ASL and empirical powers of tests T_{J},
T_{S}, T_{SY} and T* when all covariance matrices in the hypothesis
were set under heterogeneous compound symmetry pattern (CSH). We observed that
the ASL of all tests under this CSH pattern were similar formats to the ASL
obtained under UN pattern provided in Table 1. The ASL of
the tests T_{J}, T_{S} and T_{SY} were not close to
0.05 whereas that of the proposed test T* well approximate 0.05 as p and the
sample sizes increased. It can be observed that the ASL of the tests T_{S}
and T_{SY} from this table were lower than those from Table
1 for all cases considered. This indicates that the convergences of the
tests T_{S} and T_{SY} to the standard normal distribution were
very slow and not accomplished when the common covariance matrix was under CSH
and UN patterns. As displayed in this table, the empirical powers of the proposed
test T* rapidly converged to one when p and the sample sizes increased.
Table 3: 
ASL of T_{J}, T_{S}, T_{SY} and T*
under
and their empirical powers under
and
applied at α = 0.05 

ASL: The attained significance level, M_{0} and M_{1}:
The matrices defined under heterogeneous compound symmetry pattern (CSH),
α: The nominal significance level 
Table 4: 
ASL of T_{J}, T_{S}, T_{SY} and T*
under
and their empirical powers under
and
applied at α = 0.05 

ASL: The attained significance level, α: The nominal
significance level 
Table 4 reports the ASL of tests T_{J}, T_{S},
T_{SY} and T* under the common covariance matrix Σ = 2I (simple
pattern) and their empirical powers under Σ_{1} = 2I and Σ_{2}
= 1.5I. As expected, the ASL of T* and T_{J} were quite close to 0.05
for all cases considered while those of T_{S} and T_{SY} seemed
to be zero for all cases considered. The empirical powers of the proposed test
T* and T_{J} test converged to one as p and the sample sizes increased.
The convergence to one of the empirical powers of the proposed test T* was extremely
faster than that of T_{J}, especially when n_{1} = n_{2}≤40
for all p. For example, when p = n_{1} = n_{2} = 20, the empirical
powers of T* and T_{J} were 0.757 and 0.117, respectively. This indicates
that, under simple pattern, T* was reasonable test and more powerful than T_{J}
test, particularly in case of small samples.
Table 5 presents the ASL of tests T_{J}, T_{S},
T_{SY} and T* under the common covariance matrix Σ = I (simple
pattern) and empirical powers under Σ_{1} = I and a certain matrix
Σ_{2} = Diag (1,1,1,2, ..., 1,1,1,2).
Table 5: 
ASL of T_{J}, T_{S}, T_{SY} and T*
under
and their empirical powers under
and
applied at α = 0.05 

ASL: The attained significance level, α: The nominal
significance level 
Table 6: 
ASL of T_{J}, T_{S}, T_{SY} and T*
T* under and
their empirical powers under
and
when n_{2} = 2n_{1} applied at α = 0.05 

ASL: The attained significance level, α: The nominal
significance level 
As displayed in this table, the ASL of the proposed test T* and T_{J}
test were similar to those from Table 4 and reasonable approximate
0.05 for all cases of p and the sample sizes. This means that changing the scalar
σ^{2} defined in simple pattern, from σ^{2} = 2 became
σ^{2} = 1, is not effected to the convergence of the asymptotic
normality of the proposed test T* and T_{J} test. But it is greatly
effected to the convergence of the asymptotic normality of T_{S} and
T_{SY} because ASL of both tests when Σ = I were much better than
those when Σ = 2I for all case considered. However, the ASL of the tests
T_{S} and T_{SY} mainly were still not control 0.05, particularly
when the sample sizes were less than or equal to 40 for any p. As expected,
the empirical powers of the tests T_{J} and T* quickly tended to one
as p and the sample sizes increased. In addition, the proposed test T* generally
gave the higher power than T_{J} test.
We carried out additional simulations for the case that the sample sizes were
not equal (n_{1}≠n_{2}) choosing n_{2} = 2n_{1},
of four tests T_{J}, T_{S}, T_{SY} and T* under the
null hypothesis .
Corresponding empirical powers of these tests were also manipulated under the
alternative hypothesis
and Σ_{2} = Diag (1,1,1,2, ..., 1,1,1,2). The results are provided
in Table 6.
Table 6 presents that both ASL and empirical powers of these
tests were not substantially different from those given in Table
5. It appears that the tests T_{S} and T_{SY} still had
the ASL not close to 0.05, particularly for the small sample sizes, n_{1}
= 20 and n_{2} = 40 here. The proposed test statistic T* and T_{J}
test remained appropriate even the sample sizes were not the same. The empirical
powers of the proposed test T* maintained better than and converged to one faster
than those of T_{J} test.
APPLICATION
In this section, the dataset from Notterman et al.
(2001) is online at http://genomicspubs.princeton.edu
/oncology/Data/CarcinomaNormaldatasetCancerResearch.txt (last accessed:
9 October 2012). Two groups of colon tissues (adenocarcinoma and adenoma) were
examined by oligonucleotide arrays. The expression levels about 6500 human genes
were probed in 18 colon adenocarcinomas and 4 colon adenomas. We restricted
attention to a subset of all gene expressions of 100 expression levels on 4
colon adenocarcinomas and 4 colon adenomas. Thus we had n_{1} = 4, n_{2}
= 4 and p = 100. We examined whether the covariance matrices of the two groups
are equal. The data presented the observed test statistic values of T_{J}
= 0.908 and T* = 0.636. Corresponding pvalues were 0.182 and 0.524 indicating
the hypothesis of equality of such two covariance matrices of these data was
not rejected at any reasonable significance level.
CONCLUSIONS
In this study, we proposed an alternative test statistic for testing the equality
of two covariance matrices for two independent multivariate normal data with
p≥n_{i}, i = 1,2. The test statistic T* based on the consistent estimators
is introduced. Its asymptotic distribution approximately follows the standard
normal distribution as (p, n_{1}, n_{2})→∞ even if
p/n_{i}→c_{i}∈ (0, ∞), i = 1,2. The simulation
results strongly supported the performance of the proposed test statistic T*
that it accurately control size of test and not greatly affected by changing
the common covariance matrix appearing in the null hypothesis. As seen in the
simulation study, the proposed test statistic T* has the highest power among
competitive test statistics; T_{J}, which is a special case of the test
for testing the equality of several covariance matrices proposed by Schott
(2007), T_{S} and T_{SY} given by Srivastava
(2007) and Srivastava and Yanagihara (2010).
ACKNOWLEDGMENT
We would like to thank the Commission on Higher Education (CHE) of Thailand
for financial support through a grant fund under the Strategic Scholarships
Fellowships Frontier Research Networks.
APPENDIX
Most of work in this study could be viewed as an extension some results of
Srivastava (2005) and Fisher et
al. (2010). In order to proof Lemma 1 we have taken the following two
useful lemmas (lemma A1 and A2) from Srivastava (2005).
Lemma A1: Let nS∼W_{p}(Σ,n) and a_{k} = (trΣ^{k})/p,
k = 1,...,4. Then under the assumptions (A1) and (A3), unbiased and consistent
estimators of a_{2} as (p, n)→∞ is given by .
Lemma A2: Let nS∼W_{p}(Σ,n),
as defined in (2) and a_{k} = (trΣ^{k})/p, k = 1,...,4.
Then under the assumptions (A1) and (A3):
where, Φ(x) denotes the cumulative distribution function of a standard
normal random variable and:
The extensions of lemmas A1 and A2 can be obtained without proofs as follow:
Lemma A3: Let
and
i = 1,2, j = 1,...,4. Then under the assumptions (A2) and (A4), unbiased and
consistent estimators of a_{2i} as (p, n_{i}) → ∞
are given by ,
i = 1,2.
Lemma A4: Let
, i =
1,2, as defined in (1) and ,
i = 1,2, j = 1,..., 4. Then under the assumptions (A2) and (A4):
where Φ(x) denotes the cumulative distribution function of a standard
normal random variable and:
The following lemma is taken from Fisher et al.
(2010). Thus it also is presented without proof.
Lemma A5: Let nS∼W_{p}(Σ, n) and a_{k} = (trΣ^{k})/p,
k = 1,...16. Then under the assumptions (A1) and (A3), unbiased and consistent
estimators of a_{4} as (p,n)→∞ is given by
defined as:
where:
and