Shogun (toolbox)
| Original author(s) | Gunnar Rätsch Soeren Sonnenburg |
|---|---|
| Developer(s) | Soeren Sonnenburg Sergey Lisitsyn Heiko Strathmann |
| Stable release | 2.1.0 / March 17, 2013 |
| Written in | C++ |
| Operating system | Linux, Mac OS X, Cygwin |
| Type | Machine learning |
| License | GNU General Public License v3 |
| Website | http://www.shogun-toolbox.org/ |
Shogun is a free, open source toolbox written in C++. It offers numerous algorithms and data structures for machine learning problems.
Shogun is licensed under the terms of the GNU General Public License version 3 or later.
Contents |
Description [edit]
The focus of Shogun is on kernel machines such as support vector machines for regression and classification problems. Shogun also offers a full implementation of Hidden Markov models. The core of Shogun is written in C++ and offers interfaces for MATLAB, Octave, Python, R, Java, Lua, Ruby and C#. Shogun has been under active development since 1999. Today there is a vibrant user community all over the world using Shogun as a base for research and education, and contributing to the core package.
Supported algorithms [edit]
Currently Shogun supports the following algorithms:
- Support vector machines
- Dimensionality reduction algorithms, such as PCA, Kernel PCA, Locally Linear Embedding, Hessian Locally Linear Embedding, Local Tangent Space Alignment, Linear Local Tangent Space Alignment, Kernel Locally Linear Embedding, Kernel Local Tangent Space Alignment, Multidimensional Scaling, Isomap, Diffusion Maps, Laplacian Eigenmaps
- Online learning algorithms such as SGD-QN, Vowpal Wabbit
- Clustering algorithms: k-means and GMM
- Kernel Ridge Regression, Support Vector Regression
- Hidden Markov Models
- K-Nearest Neighbors
- Linear discriminant analysis
- Kernel Perceptrons.
Many different kernels are implemented, ranging from kernels for numerical data (such as gaussian or linear kernels) to kernels on special data (such as strings over certain alphabets). The currently implemented kernels for numeric data include:
- linear
- gaussian
- polynomial
- sigmoid kernels
The supported kernels for special data include:
- Spectrum
- Weighted Degree
- Weighted Degree with Shifts
The latter group of kernels allows processing of arbitrary sequences over fixed alphabets such as DNA sequences as well as whole e-mail texts.
Special features [edit]
As Shogun was developed with bioinformatics applications in mind it is capable of processing huge datasets consisting of up to 10 million samples. Shogun supports the use of pre-calculated kernels. It is also possible to use a combined kernel i.e. a kernel consisting of a linear combination of arbitrary kernels over different domains. The coefficients or weights of the linear combination can be learned as well. For this purpose Shogun offers a multiple kernel learning functionality.
References [edit]
- C.Cortes and V.N. Vapnik. Support-vector networks Machine Learning, 20(3):273—297, 1995.
- S.Sonnenburg, G.Rätsch, C.Schäfer and B.Schölkopf:, Large Scale Multiple Kernel Learning. Journal of Machine Learning Research, 7:1531-1565, July 2006, K.Bennett and E.P.-Hernandez Editors.
- T.Joachims. Making large-scale SVM learning practical In B.Schölkopf, C.J.C. Burges, and A.J. Smola, editors, Advances in Kernel Methods - Support Vector Learning, pages 169—184, Cambridge, MA, 1999. MIT Press.
- C.-C. Chang and C.-J. Lin, LIBSVM : a library for support vector machines, 2001.
- S. Sonnenburg, G. Rätsch, S. Henschel, C. Widmer, J. Behr, A. Zien, F. De Bona, A. Binder, C. Gehl and V. Franc: The SHOGUN Machine Learning Toolbox, Journal of Machine Learning Research, 11:1799−1802, June 11, 2010.
- M. Belkin and P. Niyogi. Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering. Science, 14:585–591, 2002. ISSN 10495258.
- R. Coifman and S. Lafon. Diffusion maps. Applied and Computational Harmonic Analysis, 21(1): 5–30, 2006.
- T. F. Cox and M. A. A. Cox. Multidimensional Scaling, Second Edition, volume 88 of C&H/CRC Monographs on Statistics & Applied Probability. Chapman & Hall/CRC, 2000. ISBN 1584880945.
- V. De Silva and J. B. Tenenbaum. Sparse multidimensional scaling using landmark points. Technology, pages 1–41, 2004.
- J. W. Demmel, S. C. Eisenstat, J. R. Gilbert, X. S. Li, and J. W. H. Liu. A Supernodal Approach to Sparse Partial Pivoting. SIAM Journal on Matrix Analysis and Applications, 20(3):720, 1999. ISSN 08954798.
- D. L. Donoho and C. Grimes. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proceedings of the National Academy of Sciences of the United States of America, 100(10):5591–5596, 2003.
- M. Gashler. Waffles: A Machine Learning Toolkit. Journal of Machine Learning Research, 12 (July):2383–2387, 2011.
- X. He and P. Niyogi. Locality Preserving Projections. Matrix, 16(December):153–160, 2003.
- X. He, D. Cai, S. Yan, and H.-J. Zhang. Neighborhood preserving embedding. Tenth IEEE International Conference on Computer Vision ICCV05 Volume 1, 2:1208–1213, 2005.
- R. Lehoucq, K. Maschhoff, D. Sorensen, and Yang C. Arpack numerical software library. URL http://www.caam.rice.edu/software/ARPACK/.
- R. B. Lehoucq, D. C. Sorensen, and C. Yang. Arpack users’ guide: Solution of large scale eigenvalue problems with implicitly restarted Arnoldi methods. SIAM, 1998.
- S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323–2326, 2000.
- X. Sherry Li, J. Demmel, J. Gilbert, L. Grigori, M. Shao, and I. Yamazaki. Superlu numerical software library. URL http://crd-legacy.lbl.gov/ ̃xiaoye/SuperLU/.
- J. B. Tenenbaum, V. De Silva, and J. C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319–23, 2000. ISSN 00368075.
- L.J.P van der Maaten. An introduction to dimensionality reduction using matlab, 2007.
- P. Vincent, Y. Bengio, N. Chapados, and O. Delalleau. Plearn high-performance machine learning library. URL http://plearn.berlios.de/.
- T. Zhang, J. Yang, D. Zhao, and X. Ge. Linear local tangent space alignment and application to face recognition. Neurocomputing, 70(7-9):1547–1553, 2007. ISSN 09252312.
- Z. Zhang and H. Zha. Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment. Journal of Shanghai University English Edition, 8(4):406–424, 2002.
- D. Zhao. Formulating LLE using alignment technique. Pattern Recognition, 39(11):2233–2235, 2006. ISSN 00313203.
