Feature extraction

In pattern recognition and in image processing, feature extraction is a special form of dimensionality reduction.

When the input data to an algorithm is too large to be processed and it is suspected to be notoriously redundant (much data, but not much information) then the input data will be transformed into a reduced representation set of features (also named features vector). Transforming the input data into the set of features is called feature extraction. If the features extracted are carefully chosen it is expected that the features set will extract the relevant information from the input data in order to perform the desired task using this reduced representation instead of the full size input.

General

Feature extraction involves simplifying the amount of resources required to describe a large set of data accurately. When performing analysis of complex data one of the major problems stems from the number of variables involved. Analysis with a large number of variables generally requires a large amount of memory and computation power or a classification algorithm which overfits the training sample and generalizes poorly to new samples. Feature extraction is a general term for methods of constructing combinations of the variables to get around these problems while still describing the data with sufficient accuracy.

Best results are achieved when an expert constructs a set of application-dependent features. Nevertheless, if no such expert knowledge is available general dimensionality reduction techniques may help. These include:

• Principal components analysis
• Semidefinite embedding
• Multifactor dimensionality reduction
• Multilinear subspace learning
• Nonlinear dimensionality reduction
• Isomap
• Kernel PCA
• Multilinear PCA
• Latent semantic analysis
• Partial least squares
• Independent component analysis

Image processing

It can be used in the area of image processing which involves using algorithms to detect and isolate various desired portions or shapes (features) of a digitized image or video stream. It is particularly important in the area of optical character recognition

Low-level

• Edge detection
• Corner detection
• Blob detection
• Ridge detection
• Scale-invariant feature transform

Curvature

• Edge direction, changing intensity, autocorrelation.

Image motion

• Motion detection. Area based, differential approach. Optical flow.

Shape Based

• Thresholding

• Blob extraction

• Template matching

• Hough transform
  • Lines
  • Circles/Ellipse
  • Arbitrary shapes (Generalized Hough Transform)
  • Works with any parameterizable feature (class variables, cluster detection, etc..)

Flexible methods

• Deformable, parameterized shapes
• Active contours (snakes)

Feature extraction in software

Many data analysis software packages provide for feature extraction and dimension reduction. Common numerical programming environments such as MATLAB, SciLab, NumPy and the R language provide some of the simpler feature extraction techniques (e.g. principal component analysis) via built-in commands. More specific algorithms are often available as publicly-available scripts or third-party add-ons.

See also

• Cluster analysis
• Dimensionality reduction
• Feature detection
• Feature selection
• Data mining
• Connected-component labeling
• Segmentation (image processing)

References

• Rustum, R.; Adeloye, A.; Simala, A. (2007) "Kohonen self-organising map (KSOM) extracted features for enhancing MLP-ANN prediction models of BOD₅", Water Quality and Sediment Behaviour of the Future: Predictions for the 21st Century (Proceedings of Symposium HS2005 at IUGG2007, Perugia, July 2007). IAHS Publ. 314, 181-187

External links

• JMLR Special Issue on Variable and Feature Selection
• RapidMiner and Image Processing Extension for Rapidminer (open-source tools)