Simulating Factor Structures from Continuous and Discrete Distributions using Mixture of Means within a Hierarchical Model

This paper describes a method for simulating correlated data with covariance or correlation matrix following a traditional factor structure with loadings and uniquenesses. Both discrete or continuous distributions are possible, with the only requirements being finite first and second moments and that it is possible to simulate from the distribution conditionally given a mean. The method operates by using a discrete mixture of means within a hierarchical model, where the mean of a response distribution is randomly selected from a finite number of potential means for each subject, and where each subject's means are a drawn from separate independent distributions, remaining constant over items and scaling those means by a constant if unequal loading is desired. The theoretical properties of this model are derived, including factor loadings for both the covariance and correlation matrices of generated data. These formulas used to derive exact population loadings for several common exponential family-conjugate prior models, where the response follows an exponential family distribution given a mean and the distributions of potential means follow the corresponding conjugate prior distribution. A small simulation study shows that the methods of principal axis and maximum likelihood do appear to be consistent for the factor loadings for data generated from this model, and the principal axis method appears to perform better in terms of root mean squared error.