- Abstract:
-
Traditional non-parametric statistical learning techniques are often computationally attractive, but lack the same generalization and model selection abilities as state-of-the-art Bayesian algorithms which, however, are usually computationally prohibitive. This paper makes several important contributions that allow Bayesian learning to scale to more complex, real-world learning scenarios. Firstly, we show that backfitting, a traditional non-parametric, yet highly efficient regression tool can be derived in a novel formulation within an expectation maximization(EM) framework and thus can finally be given a probabilistic interpretation. Secondly, we show that the general framework of sparse Bayesian learning and in particular the relevance vector machine (RVM), can be derived as a highly efficient algorithm using a Bayesian version of backfitting at its core. As we demonstrate on several regression and classification benchmarks, Bayesian backfitting offers a compelling alternative to current regression methods, especially when the size and dimensionality of the data challenge computational resources.
- Links To Paper
- 1st Link
- Bibtex format
- @InProceedings{EDI-INF-RR-0377,
- author = {
Aaron D'Souza
and Sethu Vijayakumar
and Stefan Schaal
},
- title = {The Bayesian Backfitting Relevance Vector Machine},
- book title = {Proc. of International Conference on Machine Learning (ICML 2004)},
- publisher = {ACM Press, New York, NY, USA},
- year = 2004,
- month = {Jul},
- volume = {69},
- pages = {31},
- doi = {http://doi.acm.org/10.1145/1015330.1015358},
- url = {http://homepages.inf.ed.ac.uk/svijayak/publications/dsouza-ICML2004.pdf},
- }
|
