An exploratory analysis using t-SNE

Abstract

Working with high dimensional data is challenging because as the number of dimensions grows, more resources and time are consumed by a data analysis or machine learning algorithms. When analyzing a dataset, one of the first tasks is plotting it to see it’s behavior, whether exist or not clustered points, or any visible pattern indicating a relationship. In a dataset with more than three dimensions, if an axis is assigned for each dimension, it is not possible to see any instance of this dataset in its original space. Thus, in order to produce visualizations, we need to reduce the dimensionality of this dataset. This work reviews three of the well-known dimensionality reduction methods and discusses their advantages and disadvantages. We collected datasets containing Brazilian social and economic indicators such as water supply and treatment, garbage collection, sewers, houses materials, municipal gross domestic product, etc. These indicators configure the dimensions of our data, therefore, depending on the dataset, we may have eight, twenty-three or more indicators. Hence, we can not visualize a point that lies in a space which each of those indicators is a dimension. Thus, we apply dimensionality reduction methods to bring them from the high-dimensional space to the two-dimensional, where we can see them. Further, we apply dimensionality reduction methods to the collected datasets, and then we discuss the results.