Some Notes on the Consequences of Pretreatment of Multivariate Data

Before performing certain statistical or machine learning techniques(e.g., regression analysis, clustering, classification, neural networks, principal components analysis, factor analysis, support vector machine, $k$-nearest neighbors, etc.), it may benecessary to preprocess and/or pretreat the data to make them suitable for theanalysis. For example, given an n × p data matrix X , which represents n multivariate observations or cases (rows) on p variables or features (columns), thecolumns and/or the rows of X may be pretreated (e.g., centered and/or scaled)before applying statistical or machine learning techniques to the data. Although centering and/or scaling the variablesdo not change the correlation structure nor the graphical representation of the data, centering and/or scaling the observations do. Inthis paper we investigate various row pretreatment methods more closely and show with theoretical proofsand numerical examples (of constructed as well as real-life data) that centering and/or scaling the rows of X changes both thegraphical structure of the observations in the multi-dimensionalspace and the correlation structure among the variables. The pretreatment of the columns and/or rows may have an impact on the output of the statistical or machine learning techniques. There may be good reasons for performing row centering and/orscaling on the data and we are not against it, but analysts who use such row operations should be aware of thegeometrical and correlation structures one has performed on the data and should alsodemonstrate that the process results in a new, more appropriate structure for their questions.