Thursday, 24 April 2014

OpenVision Library Gaussian Mixture Model Implementation

OpenVision Library Gaussian Mixture Implementation

Introduction

  • In this article we will look at gaussian mixture model.
  • Mixture Models are a type of density model which comprise a number of component functions, usually Gaussian.
  • These component functions are combined to provide a multimodal density.
  • GMM is a weighted average of gaussians where gaussian by its own mean can covariance matrix matrix
  • The mixture density is given by \begin{eqnarray*} & P(X|\mu,\Sigma,w) = \sum_k w_k \mathcal{N}(X,\mu_k,\Sigma_k) \end{eqnarray*}
  • To compute the likelyhood that a random variable is sampled from the mixture model we just need to compute the probability of individual gaussian component and then find the weighted average.
  • Instead of probability we can compute the log likelyhood of gaussian components which would simply be the log of sum of weighted exponentials. \begin{eqnarray*} & L(X|\mu,\Sigma,w) = log \sum_k exp (log w_k +log \mathcal{N}(X,\mu_k,\Sigma_k) ) \end{eqnarray*}
  • A gaussian mixture model is characterized by
    • Number of mixture components
    • Weights of mixtures
    • Mean and covariances of individual mixture components
  • Thus given the model parameters computation of probability that a vector is sampled from the gaussian mixture model is fairly simple
  • For example if we consider the following mixture models
  • The probability that vector X belongs to GMM is computed as $0.0580374$ using the method Prob
  • The probability that vector X belongs to individual mixture components is computed as $0.145093:9.5455e-17$
  • Both the above results can be verified by manual calculation or other packages etc.

    Code

  • The code for the same can be found in OpenVision git repository https://github.com/pi19404/OpenVision in files ImgMl/gaussianmixture.cpp and ImgMl/gaussianmixture.hpp files.
  • OpenVision is attempt at a opensource C/C++ Library developed in C/C++ using OpenCV,Eigen etc providing modular interface for image processing,computer vision,machine learning applications.
The PDF Version of the document can be found  below