# Correspondence analysis

Correspondence analysis (CA) is a multivariate statistical technique proposed by Herman Otto Hartley (Hirschfeld) and later developed by Jean-Paul Benzécri. It is conceptually similar to principal component analysis, but applies to categorical rather than continuous data. In a similar manner to principal component analysis, it provides a means of displaying or summarising a set of data in two-dimensional graphical form. Its aim is to display in a biplot any structure hidden in the multivariate setting of the data table. As such it is a technique from the field of multivariate ordination. Since the variant of CA described here can be applied either with a focus on the rows or on the columns it should in fact be called simple (symmetric) correspondence analysis.

It is traditionally applied to contingency tables where each cell contains either a count or a zero value. It may also be applied to binary data given the presence/absence coding represents simplified count data i.e. a 1 describes a positive count and 0 stands for a count of zero. Depending on the scores used CA preserves the chi-square distance between either the rows or the columns of the table. Because CA is a descriptive technique, it can be applied to tables regardless of a significant chisquared test. Although the $\chi ^{2}$ statistic used in inferential statistics and the chi-square distance are computationally related they should not be confused since the later works as a multivariate statistical distance measure in CA while the $\chi ^{2}$ statistic is in fact a scalar not a metric.

## Details

Like principal components analysis, correspondence analysis creates orthogonal components (or axes) and, for each item in a table i.e. for each row, a set of scores (sometimes called factor scores, see Factor analysis). Correspondence analysis is performed on the data table, conceived as matrix C of size m×n where m is the number of rows and n is the number of columns. In the following mathematical description of the method capital letters in italics refer to a matrix while letters in italics refer to vectors. Understanding the following computations requires knowledge of matrix algebra.

### Preprocessing

Before proceeding to the central computational step of the algorithm, the values in matrix C have to be transformed. First compute a set of weights for the columns and the rows (sometimes called masses), where row and column weights are given by the column and row vectors, respectively:

$w_{m}={\frac {1}{n_{C}}}C\mathbf {1} ,\quad w_{n}={\frac {1}{n_{C}}}\mathbf {1} ^{T}C.$ Here $n_{C}=\sum _{i=1}^{n}\sum _{j=1}^{m}C_{ij}$ is the sum of all cell values in matrix C, or short the sum of C, and $\mathbf {1}$ is a column vector of ones with the appropriate dimension.

Put in simple words, $w_{m}$ is just a vector whose elements are the row sums of C divided by the sum of C, and $w_{n}$ is a vector whose elements are the column sums of C divided by the sum of C.

The weights are transformed into diagonal matrices

$W_{m}=\operatorname {diag} (1/{\sqrt {w_{m}}})$ and

$W_{n}=\operatorname {diag} (1/{\sqrt {w_{n}}})$ where the diagonal elements of $W_{n}$ are $1/sqrt(w_{n})$ and those of $W_{m}$ are $1/sqrt(w_{m})$ respectively i.e. the vector elements are the inverses of the square roots of the masses. The off-diagonal elements are all 0.

Next, compute matrix $P$ by dividing $C$ by its sum

$P={\frac {1}{n_{C}}}C.$ In simple words, Matrix $P$ is just the data matrix (contingency table or binary table) transformed into portions i.e. each cell value is just the cell portion of the sum of the whole table.

Finally, compute matrix $S$ , sometimes called the matrix of standardized residuals, by matrix multiplication as

$S=W_{m}(P-w_{m}w_{n})W_{n}$ Note, the vectors $w_{m}$ and $w_{n}$ are combined in an outer product resulting in a matrix of the same dimensions as $P$ . In words the formula reads: matrix $outer(w_{m},w_{n})$ is subtracted from matrix $P$ and the resulting matrix is scaled (weighted) by the diagonal matrices $W_{m}$ and $W_{n}$ . Multiplying the resulting matrix by the diagonal matrices is equivalent to multiply the i-th row (or column) of it by the i-th element of the diagonal of $W_{m}$ or $W_{n}$ , respectively.

### Interpretation of preprocessing

The vectors $w_{m}$ and $w_{n}$ are the row and column masses or the marginal probabilities for the rows and columns, respectively. Substracting matrix $outer(w_{m},w_{n})$ from matrix $P$ is the matrix algebra version of double centering the data. Multiplying this difference by the diagonal weighting matrices results in a matrix containing weighted deviations from the origin of a vector space. This origin is defined by matrix $outer(w_{m},w_{n})$ .

In fact matrix $outer(w_{m},w_{n})$ is identical with the matrix of expected frequencies in the chi-squared test. Therefore $S$ is computationally related to the independence model used in that test. But since CA is not an inferential method the term independence model is inappropriate here.

### Orthogonal components

The table $S$ is then decomposed by a singular value decomposition as

$S=U\Sigma V^{*}\,$ where $U$ and $V$ are the left and right singular vectors of $S$ and $\Sigma$ is a square diagonal matrix with the singular values $\sigma _{i}$ of $S$ on the diagonal. $\Sigma$ is of dimension $p\leq (min(m,n)-1)$ hence $U$ is of dimension m×p and $V$ is of n×p. As orthonormal vectors $U$ and $V$ fulfill

$U^{*}U=V^{*}V=I$ .

In other words, the multivariate information that is contained in $C$ as well as in $S$ is now distributed across two (coordinate) matrices $U$ and $V$ and a diagonal (scaling) matrix $\Sigma$ . The vector space defined by them has as number of dimensions p, that is the smaller of the two values, number of rows and number of columns, minus 1.

### Inertia

While a principal component analysis may be said to decompose the (co)variance, and hence its measure of success is the amount of (co-)variance covered by the first few PCA axes - measured in eigenvalue -, a CA works with a weighted (co-)variance which is called inertia. The sum of the squared singular values is the total inertia $\mathrm {I}$ of the data table, computed as

$\mathrm {I} =\sum _{i=1}^{p}\sigma _{i}^{2}.$ The total inertia $\mathrm {I}$ of the data table can also computed directly from $S$ as

$\mathrm {I} =\sum _{i=1}^{n}\sum _{j=1}^{m}s_{ij}^{2}.$ The amount of inertia covered by the i-th set of singular vectors is $\iota _{i}$ , the principal inertia. The higher the portion of inertia covered by the first few singular vectors i.e. the larger the sum of the principal inertiae in comparison to the total inertia, the more successful a CA is. Therefore all principal inertia values are expressed as portion $\epsilon _{i}$ of the total inertia

$\epsilon _{i}=\sigma _{i}^{2}/\sum _{i=1}^{p}\sigma _{i}^{2}$ and are presented in the form of a scree plot. In fact a scree plot is just a bar plot of all principal inertia portions $\epsilon _{i}$ .

### Coordinates

To transform the singular vectors to coordinates which preserve the chisquare distances between rows or columns an additional weigthing step is necessary. The resulting coordinates are called principal coordinates in CA text books. If principal coordinates are used for rows their visualisation is called a row isometric scaling in econometrics and scaling 1 in ecology. Since the weighting includes the singular values $\Sigma$ of the matrix of standardized residuals $S$ these coordinates are sometimes referred to as singular value scaled singular vectors, or, a little bit misleading, as eigenvalue scaled eigenvectors. In fact the non-trivial eigenvectors of $SS^{*}$ are the left singular vectors $U$ of $S$ and those of $S^{*}S$ are the right singular vectors $V$ of $S$ while the eigenvalues of either of these matrices are the squares of the singular values $\Sigma$ . But since all modern algorithms for CA are based on a singular value decomposition this terminology should be avoided. In the french tradition of CA the coordinates are sometimes called (factor) scores.

Factor scores or principal coordinates for the rows of matrix C are computed by

$F_{m}=W_{m}U\Sigma$ i.e. the left singular vectors are scaled by the inverse of the square roots of the row masses and by the singular values. Because principal coordinates are computed using singular values they contain the information about the spread between the rows (or columns) in the original table. Computing the euclidean distances between the entities in principal coordinates results in values that equal their chisquare distances which is the reason why CA is said to "preserve chisquare distances".

Compute principal coordinates for the columns by

$F_{n}=W_{n}V\Sigma .$ To represent the result of CA in a proper biplot, those categories which are not plotted in principal coordinates, i.e. in chisquare distance preserving coordinates, should be plotted in so called standard coordinates. When computing standard coordinates the singular values are omitted which is a direct result of applying the biplot rule by which one of the two sets of singular vector matrices must be scaled by singular values raised to the power of zero i.e. multiplied by one i.e. be computed by omitting the singular values if the other set of singular vectors have been scaled by the singuar values. This reassures the existence of a inner product between the two sets of coordinates i.e. it leads to meaningful interpretations of their spatial relations in a biplot.

In practical terms one can think of the standard coordinates as the vertices of the vector space in which the set of principal coordinates "exists". The standard coordinates for the rows are

$G_{m}=W_{m}U$ and those for the columns are

$G_{n}=W_{n}V$ Note that a scaling 1 biplot in ecology implies the rows to be in principal and the columns to be in standard coordinates while scaling 2 implies the rows to be in standard and the columns to be in principal coordinates. I.e. scaling 1 implies a biplot of $F_{m}$ together with $G_{n}$ while scaling 2 implies a biplot of $F_{n}$ together with $G_{m}$ .

## Graphical representation of result

The visualisation of a CA result always starts with displaying a screeplot of the principal inertia.

The actual ordination is presented in a graph which could - at first look - be confused with a complicated scatter plot. In fact it consists of two scatter plots printed one upon the other, one set of points for the rows and one for the columns. But being a biplot a clear interpretation rule relates the two coordinate matrices used.

Usually the first two dimensions of the CA solution are plotted because they encompass the maximum of information about the data table that can be displayed in 2D although other combinations of dimensions may be investigated by a biplot. A biplot is in fact a low dimensional mapping of a part of the information contained in the original table.

As a rule of thumb that set (rows or columns) which should be analysed with respect to its composition as measured by the other set is displayed in principal coordinates while the other set is displayed in standard coordinates. E.g. a table displaying voting districts in rows and political parties in columns with the cells containing the counted votes may be displayed with the districts (rows) in principal coordinates when the focus is on ordering districts according to similar voting.

## Extensions and applications

Several variants of CA are available, including detrended correspondence analysis (DCA) and canonical correspondence analysis (CCA). The later (CCA) is the method to use, when there is information about possible causes for the similarities between the investigated entities. The extension of correspondence analysis to many categorical variables is called multiple correspondence analysis. An adaptation of correspondence analysis to the problem of discrimination based upon qualitative variables (i.e., the equivalent of discriminant analysis for qualitative data) is called discriminant correspondence analysis or barycentric discriminant analysis.

In the social sciences, correspondence analysis, and particularly its extension multiple correspondence analysis, was made known outside France through French sociologist Pierre Bourdieu's application of it.

## Implementations

• The data visualization system Orange include the module: orngCA.
• The statistical programming language R includes several packages, which offer a function for (simple symmetric) correspondence analysis. Using the R notation [package_name::function_name] the packages and respective functions are: ade4::dudi.coa(), ca::ca() , ExPosition::epCA(), FactoMineR::CA(), MASS::corresp(), vegan::cca(). The easiest approach for beginners is ca::ca() as there is an extensive text book accompanying that package.
• The Freeware PAST (PAleontological STatistics) offers (simple symmetric) correspondence analysis via the menu "Multivariate/Ordination/Correspondence (CA)".