Wishart, matrix Gamma, Hotelling T-squared, Wilks' Lambda distributions
July 13, 2021 5 min read
In this post I'll briefly cover the multivariate analogues of gamma-distribution-related univariate statistical distributions.
Wishart distribution is a multivariate analogue of Chi-square distribution.
Suppose that you have a multivariate normal distribution, but you don’t know its mean and covariance matrix.
Denote a gaussian vector, where are individual one-dimensional gaussian random variables with 0 expectation, and their covariance matrix is unknown.
Similarly to how we tried to infer sample variance from the empirical data in univariate case (and found out that it is chi-square-distributed), we might want to try to infer the covariance matrix from sample points of our multivariate normal distribution.
Our observations then are i.i.d -vector random variables .
Then a random matrix, formed by sum of outer products of those random variables, has Wishart distribution with n degrees of freedom and covariance matrix .
Probability density function of this distribution is:
, where is multivariate gamma-function, and and are determinants of the respective matrices.
TODO: meaning of inverse matrix, trace, ratio of determinants
TODO: use in Bayesian statistics as conjugate prior
Matrix Gamma distribution
Matrix Gamma distribution to Wishart distribution in multivariate case is what gamma distribution is to Chi-square distribution in univariate case.
Not much to say about it, really, because it is not very useful on its own.
Hotelling T-squared distribution
Hotelling T-squared distribution is often used as a multivariate analogue of t-Student.
Remember, how in case of univariate normal distribution, we considered a random variable , where is sample mean, - true (distribution) mean, and is sample variance?
We showed that its square has a Snedecor-Fisher F distribution.
In case of multivariate normal distribution we generalize sample variance with a sample covariance matrix (which approximates the real covariance matrix ). While the sample variance was chi-squared-distributed, sample covariance matrix is Wishart-distributed (a matrix generalization of the chi-square distribution).
In multivariate case we consider a quadratic form . Note, how this random variable corresponds to in univariate case - we replace division by variance with multiplication by inverse sample covariance matrix.
It turns out that, again, sample means vector is independent from the sample covariance matrix (a result, similar to univariate Cochran’s theorem).
It also turns out that, again:
, where is a p-variate normal distribution with vector of means and covariance matrix;
, where is a p-dimensional Wishart distribution with degrees of freedom and covariance matrix;
, where n - number of sample points, p - dimensionality of the data, - Fisher-Snedecor distribution with p and n-p degrees of freedom.
Hotelling distribution is defined as , so that .
Wilks’ Lambda distribution
Wilks’ Lambda is another distribution, very similar to Snedecor-Fisher’s F distribution.
As F distribution is a ratio of two chi-square distributions, Wilks’ Lambda distribution is a ratio of determinants of two Wishart-distributed random matrices (note that the sum of two independent Wishart-distributed matrices with and degrees of freedom and identical covariance matrices is a Wishart-distributed matrix with degrees of freedom):
, , both independent,
Written by Boris Burkov who lives in Moscow, Russia and Cambridge, UK, loves to take part in development of cutting-edge technologies, reflects on how the world works and admires the giants of the past. You can follow me on Telegram