Relevance vector machine: Difference between revisions
No edit summary |
m Added a citation where "citation needed" was tagged |
||
Line 6: | Line 6: | ||
:<math>k(\mathbf{x},\mathbf{x'}) = \sum_{j=1}^N \frac{1}{\alpha_j} \varphi(\mathbf{x},\mathbf{x}_j)\varphi(\mathbf{x}',\mathbf{x}_j) </math> |
:<math>k(\mathbf{x},\mathbf{x'}) = \sum_{j=1}^N \frac{1}{\alpha_j} \varphi(\mathbf{x},\mathbf{x}_j)\varphi(\mathbf{x}',\mathbf{x}_j) </math> |
||
where <math>\varphi</math> is the [[kernel function]] (usually Gaussian),<math>\alpha_j</math>'s as the variances of the prior on the weight vector |
where <math>\varphi</math> is the [[kernel function]] (usually Gaussian),<math>\alpha_j</math>'s as the variances of the prior on the weight vector |
||
<math>w \sim N(0,\alpha^{-1}I)</math> ,and <math>\mathbf{x}_1,\ldots,\mathbf{x}_N</math> are the input vectors of the [[training set]].{{ |
<math>w \sim N(0,\alpha^{-1}I)</math> ,and <math>\mathbf{x}_1,\ldots,\mathbf{x}_N</math> are the input vectors of the [[training set]].<ref>{{cite thesis |
||
|type=Ph.D. |
|||
|last=Candela |
|||
|first=Joaquin Quiñonero |
|||
|date=2004 |
|||
|title=Learning with Uncertainty - Gaussian Processes and Relevance Vector Machines |
|||
|publisher=Technical University of Denmark |url=http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/3237/pdf/imm3237.pdf |chapter=Sparse Probabilistic Linear Models and the RVM |
|||
|access-date=April 22, 2016 |
|||
}}</ref> |
|||
Compared to that of [[support vector machine]]s (SVM), the Bayesian formulation of the RVM avoids the set of free parameters of the SVM (that usually require cross-validation-based post-optimizations). However RVMs use an [[expectation maximization]] (EM)-like learning method and are therefore at risk of local minima. This is unlike the standard [[sequential minimal optimization]] (SMO)-based algorithms employed by [[Support vector machine|SVM]]s, which are guaranteed to find a global optimum (of the convex problem). |
Compared to that of [[support vector machine]]s (SVM), the Bayesian formulation of the RVM avoids the set of free parameters of the SVM (that usually require cross-validation-based post-optimizations). However RVMs use an [[expectation maximization]] (EM)-like learning method and are therefore at risk of local minima. This is unlike the standard [[sequential minimal optimization]] (SMO)-based algorithms employed by [[Support vector machine|SVM]]s, which are guaranteed to find a global optimum (of the convex problem). |
Revision as of 14:22, 22 April 2016
Part of a series on |
Machine learning and data mining |
---|
In mathematics, a Relevance Vector Machine (RVM) is a machine learning technique that uses Bayesian inference to obtain parsimonious solutions for regression and probabilistic classification.[1] The RVM has an identical functional form to the support vector machine, but provides probabilistic classification.
It is actually equivalent to a Gaussian process model with covariance function:
where is the kernel function (usually Gaussian),'s as the variances of the prior on the weight vector ,and are the input vectors of the training set.[2]
Compared to that of support vector machines (SVM), the Bayesian formulation of the RVM avoids the set of free parameters of the SVM (that usually require cross-validation-based post-optimizations). However RVMs use an expectation maximization (EM)-like learning method and are therefore at risk of local minima. This is unlike the standard sequential minimal optimization (SMO)-based algorithms employed by SVMs, which are guaranteed to find a global optimum (of the convex problem).
The relevance vector machine is patented in the United States by Microsoft.[3]
See also
- Kernel trick
- Platt scaling: turns an SVM into a probability model
References
- ^ Tipping, Michael E. (2001). "Sparse Bayesian Learning and the Relevance Vector Machine". Journal of Machine Learning Research. 1: 211–244.
- ^ Candela, Joaquin Quiñonero (2004). "Sparse Probabilistic Linear Models and the RVM". Learning with Uncertainty - Gaussian Processes and Relevance Vector Machines (PDF) (Ph.D.). Technical University of Denmark. Retrieved April 22, 2016.
- ^ US 6633857, Michael E. Tipping, "Relevance vector machine"
Software
- dlib C++ Library
- The Kernel-Machine Library
- rvmbinary:R package for binary classification
- scikit-rvm
- fast-scikit-rvm , rvm tutorial