Principal component regression

Regression analysis
Models
Linear regression Simple regression Ordinary least squares Polynomial regression General linear model
Generalized linear model Discrete choice Logistic regression Multinomial logit Mixed logit Probit Multinomial probit Ordered logit Ordered probit Poisson
Multilevel model Fixed effects Random effects Mixed model
Nonlinear regression Nonparametric Semiparametric Robust Quantile Isotonic Principal components Least angle Local Segmented
Errors-in-variables
Estimation
Least squares Ordinary least squares Linear (math) Partial Total Generalized Weighted Non-linear Iteratively reweighted Ridge regression LASSO
Least absolute deviations Bayesian Bayesian multivariate
Background
Regression model validation Mean and predicted response Errors and residuals Goodness of fit Studentized residual Gauss–Markov theorem
Statistics portal

In statistics, principal component regression (PCR) is a regression analysis that uses principal component analysis when estimating regression coefficients. It is a procedure used to overcome problems which arise when the exploratory variables are close to being collinear.

In PCR instead of regressing the dependent variable on the independent variables directly, the principal components of the independent variables are used. One typically only uses a subset of the principal components in the regression, making a kind of regularized estimation.

Often the principal components with the highest variance are selected. However, the low-variance principal components may also be important, — in some cases even more important.

The principle

PCR (principal components regression) is a regression method that can be divided into three steps:^{[citation needed]}

1. The first step is to run a principal components analysis on the table of the explanatory variables,
2. The second step is to run an ordinary least squares regression (linear regression) on the selected components: the factors that are most correlated with the dependent variable will be selected
3. Finally the parameters of the model are computed for the selected explanatory variables.

See also

• Canonical correlation
• Deming regression
• Multilinear subspace learning
• Partial least squares regression
• Principal component analysis
• Total sum of squares

References

• R. Kramer, Chemometric Techniques for Quantitative Analysis, (1998) Marcel-Dekker, ISBN 0-8247-0198-4.