Thursday, 24 April 2014

Gaussian Multivariate Distribution -Part 1

Multi Variate Gaussian Distribution - Part 1


In this article we will look at the multivariate gaussian distribution.
  • The mutivariate gaussian distribution is parameterized by mean vector $\boldsymbol\mu$ and covariance matrix $\boldsymbol\Sigma$. \begin{eqnarray*} & f_{\mathbf x}(x_1,\ldots,x_k) =\frac{1}{\sqrt{(2\pi)^k|\boldsymbol\Sigma|}} \exp\left(-\frac{1}{2}({\mathbf x}-{\boldsymbol\mu})^T{\boldsymbol\Sigma}^{-1}({\mathbf x}-{\boldsymbol\mu})\right), \\ & where\\ & \text{$\boldsymbol\mu$ is a Nx1 mean vector }\\ & \text{$\boldsymbol\Sigma$ is NxN matrix covariance matrix} \\ & \text{$|\boldsymbol\Sigma|$ is the determinant of $\boldsymbol\Sigma$} \\ \end{eqnarray*}
  • The mutivariate gaussian is also used as probability density function of the vector ${X}$
  • A naive way of implementation is to directly perform the computation as in the equation to get the result.
  • A Gaussian is defined
  • Since ill be using a lot of opencv conversion routines are defined for Opencv cv::Mat data structure to Eigen::MatrixXf data structure.
  • when loading the covarince matrix ,the determinant,inverse etc are also computed which would be later required for probability calculation
  • The probability computation is simply writing out the formula
  • The covariance matrix is postive semidefinate and symmteric
  • The Cholesky decomposition of a symmetric, positive definite matrix $\boldsymbol\Sigma$ decom- poses A into a product of a lower triangular matrix L and its transpose.
  • Thus the product \begin{eqnarray*} & \boldsymbol\Sigma=LL^T \\ & \boldsymbol\Sigma^{-1}=(LL^T)^{-1}=L^{-T}L^{-1} \\ & {\mathbf y}^T{\boldsymbol\Sigma}^{-1}{\mathbf y} \\ & {\mathbf y}^T{\boldsymbol\Sigma}^{-1}{\mathbf y} \\ & {\mathbf y}^T{L^{-T}L^{-1}}{\mathbf y} \\ & \left({L^{-1}\mathbf y}\right)^T\left({L^{-1}\mathbf y}\right) \end{eqnarray*}
  • thus we can only compute $\left({L^{-1}\mathbf y}\right)$ and take its tranpose
  • Thus we can perform the Cholskey factorization of the covariance matrix to find $\mathbf L$
  • Then we take the inverse of $\mathbf L $


    The code can be found at git repository in files ImgML/gaussian.hpp and ImgML.gaussian.cpp files.
The PDF Version of the document can be found below