Functional principal component analysis
Functional principal component analysis (FPCA) is a statistical method for investigating the dominant modes of variation of functional data. Using this method, a random function is represented in the eigenbasis, which is an orthonormal basis of the Hilbert space L2 that consists of the eigenfunctions of the autocovariance operator. FPCA represents functional data in the most parsimonious way, in the sense that when using a fixed number of basis functions, the eigenfunction basis explains more variation than any other basis expansion. FPCA can be applied for representing random functions, or in functional regression and classification.
By the Karhunen–Loève theorem, one can express the centered process in the eigenbasis,
is the principal component associated with the k-th eigenfunction , with the properties
The centered process is then equivalent to ξ1, ξ2, .... A common assumption is that X can be represented by only the first few eigenfunctions (after subtracting the mean function), i.e.
Interpretation of eigenfunctions
The first eigenfunction depicts the dominant mode of variation of X.
The k-th eigenfunction is the dominant mode of variation orthogonal to , , ... , ,
Let Yij = Xi(tij) + εij be the observations made at locations (usually time points) tij, where Xi is the i-th realization of the smooth stochastic process that generates the data, and εij are identically and independently distributed normal random variable with mean 0 and variance σ2, j = 1, 2, ..., mi. To obtain an estimate of the mean function μ(tij), if a dense sample on a regular grid is available, one may take the average at each location tij:
Then the estimate of the covariance function is obtained by averaging (in the dense case) or smoothing (in the sparse case) the raw covariances
Note that the diagonal elements of Gi should be removed because they contain measurement error.
In practice, is discretized to an equal-spaced dense grid, and the estimation of eigenvalues λk and eigenvectors vk is carried out by numerical linear algebra. The eigenfunction estimates can then be obtained by interpolating the eigenvectors
Let be a smoothed version of the diagonal elements Gi(tij, tij) of the raw covariance matrices. Then is an estimate of (G(t, t) + σ2). An estimate of σ2 is obtained by
- if otherwise
If the observations Xij, j=1, 2, ..., mi are dense in 𝒯, then the k-th FPC ξk can be estimated by numerical integration, implementing
FPCA can be applied for displaying the modes of functional variation, in scatterplots of FPCs against each other or of responses against FPCs, for modeling sparse longitudinal data, or for functional regression and classification, e.g., functional linear regression. Scree plots and other methods can be used to determine the number of included components. Functional Principal component analysis has varied applications in time series analysis. Nowadays, this methodology is being adapted from traditional multi-variate techniques to carry out analysis on financial data sets such as stock market indices, generation of implied volatility graphs and so on. A very nice example of the advantages of the functional approach is the Smoothed FPCA (SPCA), proposed by Silverman  and studied by Pezzulli and Silverman  that enables direct combination of the FPCA analysis together with a general smoothing approach that makes the use of the information stored in some linear diﬀerential operators possible. An important application of the FPCA already known from multivariate PCA, is motivated by the Karhunen-Loève decomposition of a random function to the set of functional parameters – factor functions and corresponding factor loadings (scalar random variables). This application is much more important than in the standard multivariate PCA since the distribution of the random function is in general too complex to be directly analyzed and the Karhunen-Loève decomposition reduces the analysis to the interpretation of the factor functions and the distribution of scalar random variables. Due to dimensionality reduction as well as its accuracy to represent data, there is a wide scope for further developments of functional principal component techniques in the financial field.
Connection with principal component analysis
The following table shows a comparison of various elements of principal component analysis (PCA) and FPCA. The two methods are both used for dimensionality reduction. In implementations, FPCA uses a PCA step.
However, PCA and FPCA differ in some critical aspects. First, the order of multivariate data in PCA can be permuted, which has no effect on the analysis, but the order of functional data carries time or space information and cannot be reordered. Second, the spacing of observations in FPCA matters, while there is no spacing issue in PCA. Third, regular PCA does not work for high-dimensional data without regularization, while FPCA has a built-in regularization due to the smoothness of the functional data and the truncation to a finite number of included components.
|Element||In PCA||In FPCA|
- Jones, M. C.; Rice, J. A. (1992). "Displaying the Important Features of Large Collections of Similar Curves". The American Statistician. 46 (2): 140. doi:10.1080/00031305.1992.10475870.
- Yao, F.; Müller, H. G.; Wang, J. L. (2005). "Functional linear regression analysis for longitudinal data". The Annals of Statistics. 33 (6): 2873. arXiv:math/0603132. doi:10.1214/009053605000000660.
- Yao, F.; Müller, H. G.; Wang, J. L. (2005). "Functional Data Analysis for Sparse Longitudinal Data". Journal of the American Statistical Association. 100 (470): 577. doi:10.1198/016214504000001745.
- Staniswalis, J. G.; Lee, J. J. (1998). "Nonparametric Regression Analysis of Longitudinal Data". Journal of the American Statistical Association. 93 (444): 1403. doi:10.1080/01621459.1998.10473801.
- Rice, John; Silverman, B. (1991). "Estimating the Mean and Covariance Structure Nonparametrically When the Data are Curves". Journal of the Royal Statistical Society. Series B (Methodological). Wiley. 53 (1): 233–243.
- "PACE: Principal Analysis by Conditional Expectation".
- "fdapace: Functional Data Analysis and Empirical Dynamics".
- Hall, P.; Müller, H. G.; Wang, J. L. (2006). "Properties of principal component methods for functional and longitudinal data analysis". The Annals of Statistics. 34 (3): 1493. arXiv:math/0608022. doi:10.1214/009053606000000272.
- Li, Y.; Hsing, T. (2010). "Uniform convergence rates for nonparametric regression and principal component analysis in functional/longitudinal data". The Annals of Statistics. 38 (6): 3321. arXiv:1211.2137. doi:10.1214/10-AOS813.
- Madrigal, Pedro; Krajewski, Paweł (2015). "Uncovering correlated variability in epigenomic datasets using the Karhunen-Loeve transform". BioData Mining. 8. doi:10.1186/s13040-015-0051-7.
- Functional Data Analysis with Applications in Finance by Michal Benko