Independent vector analysis using semi-parametric density estimation via multivariate entropy maximization

Abstract

Blind source separation (BSS) is an active area of research in statistical signal processing due to its numerous applications, such as analysis of medical imaging data, wireless communications, and image processing. Due to the wide use of multi-sensor technology, analysis of multiple datasets is at the heart of many challenging engineering problems. This motivates the development of the field of joint blind source separation (JBSS), which extends the classical BSS to simultaneously resolve several BSS problems by assuming statistical dependence between latent sources across mixtures. Independent component analysis (ICA) is a widely used BSS method that can uniquely achieve source recovery, subject to only scaling and permutation ambiguities, through the assumption of statistical independence on the part of the latent sources. Although ICA is one of the most commonly used, it can only decompose a single dataset. This has driven the development of independent vector analysis (IVA), a recent generalization of ICA to multiple datasets that can achieve improved performance over performing ICA on each dataset separately by exploiting dependencies across datasets. Though both ICA and IVA algorithms cast in the maximum likelihood (ML) framework such that all available types of diversity are taken into account simultaneously through the use of general density models for the latent multivariate sources, they often deviate from their theoretical optimality properties due to improper estimation of the probability density function (PDF). Therefore, in order to guarantee the effectiveness of IVA algorithms, an efficient density estimation method is required. In this dissertation, we present a multivariate density estimation technique based on the maximum entropy principle (MEP) that jointly uses global and local multidimensional measuring functions to provide flexible PDFs while keeping the complexity low by integrating into the proposed algorithm a multidimensional Monte-Carlo (MC) integration technique. Finally, we derive a new IVA algorithm, which takes advantage of the accurate estimation capability of the proposed density estimation method to greatly improve separation performance from a wide range of distributions. We use numerical experiments to demonstrate the superior performance over widely used algorithms.