ALOPEX

From Wikipedia, the free encyclopedia
Jump to: navigation, search

ALOPEX (an acronym from "ALgorithms Of Pattern EXtraction") is a correlation based machine learning algorithm first proposed by Tzanakou and Harth in 1974.

Principle[edit]

In machine learning, the goal is to train a system to minimize a cost function or (referring to ALOPEX) a response function. Many training algorithms, such as backpropagation, have an inherent susceptibility to getting "stuck" in local minima or maxima of the response function. ALOPEX uses a cross-correlation of differences and a stochastic process to overcome this in an attempt to reach the absolute minimum (or maximum) of the response function.

Method[edit]

ALOPEX, in its simplest form is defined by an updating equation:

Where:

  • is the iteration or time-step.
  • is the difference between the current and previous value of system variable at iteration .
  • is the difference between the current and previous value of the response function at iteration .
  • is the learning rate parameter minimizes and maximizes

Discussion[edit]

Essentially, ALOPEX changes each system variable based on a product of: the previous change in the variable , the resulting change in the cost function , and the learning rate parameter . Further, to find the absolute minimum (or maximum), the stochastic process (Gaussian or other) is added to stochastically "push" the algorithm out of any local minima.

References[edit]

  • Harth, E., & Tzanakou, E. (1974) Alopex: A stochastic method for determining visual receptive fields. Vision Research, 14:1475-1482. Abstract from ScienceDirect