Contributions on latent projections for Gaussian process modeling

Abstract

Projecting data to a latent space is a routine procedure in machine learning. One of the incentives to do such transformations is the manifold hypothesis, which states that most data sampled from empirical processes tend to be inside a lower-dimensional space. Since this smaller representation is not visible in the dataset, probabilistic machine learning techniques can accurately propagate uncertainties in the data to the latent representation. In particular, Gaussian processes (GP) are a family of probabilistic kernel methods that researchers have successfully applied to regression and dimensionality reduction tasks. However, for dimensionality reduction, efficient and deterministic variational inference exists only for a minimal set of kernels. As such, I propose the unscented Gaussian process latent variable model (UGPLVM), an alternative inference method for Bayesian Gaussian process latent variable models that uses the unscented transformation to permit the use of arbitrary kernels while remaining sample efficient. For regression with GP models, the compositional deep Gaussian process (DGP) is a popular model that uses successive mappings to latent spaces to alleviate the burden of choosing a kernel function. However, that is not the only DGP construction possible. In this dissertation, I propose another DGP construction in which each layer controls the smoothness of the next layer, instead of directly composing layer outputs into layer inputs. This model is called deep Mahalanobis Gaussian process (DMGP), and it is based on previous literature on the integration of Mahalanobis kernel hyperparameters and, thus, incorporates the idea of locally linear projections. Both proposals use deterministic variational inference while maintaining the same results and scalability as non-deterministic methods in various experimental tasks. The experiments for UGPLVM cover dimensionality reduction and simulation of dynamic systems with uncertainty propagation, and, for DMGP, they cover regression tasks with synthetic and empirical datasets.