Radial basis function

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

A radial basis function (RBF) is a real-valued function whose value depends only on the distance between the input and some fixed point, either the origin, so that , or some other fixed point , called a center, so that . Any function that satisfies the property is a radial function. The distance is usually Euclidean distance, although other metrics are sometimes used. They are often used as a collection which forms a basis for some function space of interest, hence the name.

Sums of radial basis functions are typically used to approximate given functions. This approximation process can also be interpreted as a simple kind of neural network; this was the context in which they were originally applied to machine learning, in work by David Broomhead and David Lowe in 1988,[1][2] which stemmed from Michael J. D. Powell's seminal research from 1977.[3][4][5] RBFs are also used as a kernel in support vector classification.[6] The technique has proven effective and flexible enough that radial basis functions are now applied in a variety of engineering applications.[7][8]

Definition[edit]

A radial function is a function . When paired with a metric on a vector space a function is said to be a radial kernel centered at . A Radial function and the associated radial kernels are said to be radial basis functions if, for any set of nodes

  • The kernels are linearly independent (for example in is not a radial basis function)
  • The kernels form a basis for a Haar Space, meaning that the interpolation matrix

is non-singular. [9] [10]

Examples[edit]

Commonly used types of radial basis functions include (writing and using to indicate a shape parameter that can be used to scale the input of the radial kernel[11]):

  • Infinitely Smooth RBFs

These radial basis functions are from and are strictly positive definite functions[12] that require tuning a shape parameter

  • Gaussian:
A Gaussian function for several choices of .
A plot of the scaled Bump function with several choices of .
  • Multiquadric:
  • Inverse quadratic:
  • Inverse multiquadric:
  • Polyharmonic spline:
    *For even-degree polyharmonic splines , to avoid numerical problems at where , the computational implementation is often written as .
  • Thin plate spline (a special polyharmonic spline):
  • Compactly Supported RBFs

These RBFs are compactly supported and thus are non-zero only within a radius of , and thus have sparse differentiation matrices


Approximation[edit]

Radial basis functions are typically used to build up function approximations of the form

where the approximating function is represented as a sum of radial basis functions, each associated with a different center , and weighted by an appropriate coefficient The weights can be estimated using the matrix methods of linear least squares, because the approximating function is linear in the weights .

Approximation schemes of this kind have been particularly used[citation needed] in time series prediction and control of nonlinear systems exhibiting sufficiently simple chaotic behaviour, 3D reconstruction in computer graphics (for example, hierarchical RBF and Pose Space Deformation).

RBF Network[edit]

Two unnormalized Gaussian radial basis functions in one input dimension. The basis function centers are located at and .

The sum

can also be interpreted as a rather simple single-layer type of artificial neural network called a radial basis function network, with the radial basis functions taking on the role of the activation functions of the network. It can be shown that any continuous function on a compact interval can in principle be interpolated with arbitrary accuracy by a sum of this form, if a sufficiently large number of radial basis functions is used.

The approximant is differentiable with respect to the weights . The weights could thus be learned using any of the standard iterative methods for neural networks.

Using radial basis functions in this manner yields a reasonable interpolation approach provided that the fitting set has been chosen such that it covers the entire range systematically (equidistant data points are ideal). However, without a polynomial term that is orthogonal to the radial basis functions, estimates outside the fitting set tend to perform poorly.[citation needed]

See also[edit]

References[edit]

  1. ^ Radial Basis Function networks Archived 2014-04-23 at the Wayback Machine
  2. ^ Broomhead, David H.; Lowe, David (1988). "Multivariable Functional Interpolation and Adaptive Networks" (PDF). Complex Systems. 2: 321–355. Archived from the original (PDF) on 2014-07-14.
  3. ^ Michael J. D. Powell (1977). "Restart procedures for the conjugate gradient method". Mathematical Programming. 12 (1): 241–254. doi:10.1007/bf01593790.
  4. ^ Sahin, Ferat (1997). A Radial Basis Function Approach to a Color Image Classification Problem in a Real Time Industrial Application (PDF) (M.Sc.). Virginia Tech. p. 26. Radial basis functions were first introduced by Powell to solve the real multivariate interpolation problem.
  5. ^ Broomhead & Lowe 1988, p. 347: "We would like to thank Professor M.J.D. Powell at the Department of Applied Mathematics and Theoretical Physics at Cambridge University for providing the initial stimulus for this work."
  6. ^ VanderPlas, Jake (6 May 2015). "Introduction to Support Vector Machines". [O'Reilly]. Retrieved 14 May 2015.
  7. ^ Buhmann, Martin Dietrich (2003). Radial basis functions : theory and implementations. Cambridge University Press. ISBN 978-0511040207. OCLC 56352083.
  8. ^ Biancolini, Marco Evangelos (2018). Fast radial basis functions for engineering applications. Springer International Publishing. ISBN 9783319750118. OCLC 1030746230.
  9. ^ Fasshauer, Gregory E. (2007). Meshfree Approximation Methods with MATLAB. Singapore: World Scientific Publishing Co. Pte. Ltd. pp. 17–25. ISBN 9789812706331.
  10. ^ Wendland, Holger (2005). Scattered Data Approximation. Cambridge: Cambridge University Press. pp. 11, 18–23, 64–66. ISBN 0521843359.
  11. ^ Fasshauer, Gregory E. (2007). Meshfree Approximation Methods with MATLAB. Singapore: World Scientific Publishing Co. Pte. Ltd. p. 37. ISBN 9789812706331.
  12. ^ Fasshauer, Gregory E. (2007). Meshfree Approximation Methods with MATLAB. Singapore: World Scientific Publishing Co. Pte. Ltd. pp. 37–45. ISBN 9789812706331.

Further reading[edit]

  • Hardy, R.L. (1971). "Multiquadric equations of topography and other irregular surfaces". Journal of Geophysical Research. 76 (8): 1905–1915. Bibcode:1971JGR....76.1905H. doi:10.1029/jb076i008p01905.
  • Hardy, R.L. (1990). "Theory and applications of the multiquadric-biharmonic method, 20 years of Discovery, 1968 1988". Comp. Math Applic. 19 (8/9): 163–208. doi:10.1016/0898-1221(90)90272-l.
  • Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (2007), "Section 3.7.1. Radial Basis Function Interpolation", Numerical Recipes: The Art of Scientific Computing (3rd ed.), New York: Cambridge University Press, ISBN 978-0-521-88068-8
  • Sirayanone, S., 1988, Comparative studies of kriging, multiquadric-biharmonic, and other methods for solving mineral resource problems, PhD. Dissertation, Dept. of Earth Sciences, Iowa State University, Ames, Iowa.
  • Sirayanone, S.; Hardy, R.L. (1995). "The Multiquadric-biharmonic Method as Used for Mineral Resources, Meteorological, and Other Applications". Journal of Applied Sciences and Computations. 1: 437–475.