On the Relationship between Factor Analysis and Principal Component Analysis in High-Dimensions?
Kentaro Hayashi
University of Hawaii at Manoa, USA
hayashik@hawaii.edu
Ke-Hai Yuan
University of Notre Dame, USA
kyuan@nd.edu
Abstract. This article reviews the relationship between loadings from factor analysis (FA) and those from principal component analysis (PCA) when the number of variables p is large. While FA and PCA are substantively different methodologies, the two loading matrices are often close to each other. After defining how to measure the degree of closeness between the two loading matrices under rotational indeterminacy, the article reviews the conditions under which the two loading matrices agree with each other. Two well-known conditions for characterizing the two sets of loadings are given by Guttman (1956) and by Schneeweiss (1997), and they are further refined by Krijnen (2006). The relationship between these conditions is discussed, and results are provided showing that the two loading are closely related when p is large. Estimation methods are described to deal with conditions when sparsity does not hold in the covariance matrix and when p is not much greater than the sample size. Also, the problem of an increased bias in eigenvalues of the covariance matrix as p increases is also noted.
Keywords: Bias in eigenvalues • Canonical correlation • Matrix norm.
DOI: https://doi.org/10.35566/isdsa2019c9
To buy this chapter or the book, use the form below