Informatics Report Series


Report   

EDI-INF-RR-0375


Related Pages

Report (by Number) Index
Report (by Date) Index
Author Index
Institute Index

Home
Title:Incremental Online Learning in High Dimensions
Authors: Sethu Vijayakumar ; Aaron D'Souza ; Stefan Schaal
Date:Dec 2005
Publication Title:Neural Computation
Publisher:MIT Press
Publication Type:Journal Article Publication Status:Published
Volume No:17 Page Nos:2602-2634
Abstract:
Locally weighted projection regression (LWPR) is a new algorithm for incremental non-linear function approximation in high dimensional spaces with redundant and irrelevant input dimensions. At its core, it employs nonparametric regression with locally linear models. In order to stay computationally ecient and numerically robust, each local model performs the regression analysis with a small number of univariate regressions in selected directions in input space in the spirit of partial least squares regression. We discuss when and how local learning techniques can successfully work in high dimensional spaces and review the various techniques for local dimensionality reduction before finally deriving the LWPR algorithm. The properties of LWPR are that it i) learns rapidly with second order learning methods based on incremental training, ii) uses statistically sound stochastic leave-one-out cross validation for learning without the need to memorize training data, iii) adjusts its weighting kernels based only on local information in order to minimize the danger of negative interference of incremental learning, iv) has a computational complexity that is linear in the number of inputs, and v) can deal with a large number of - possibly redundant - inputs, as shown in various empirical evaluations with up to 90 dimensional data sets. For a probabilistic interpretation, predictive variance and condence intervals are derived. To our knowledge, LWPR is the rst truly incremental spatially localized learning method that can successfully and eciently operate in very high dimensional spaces.
Links To Paper
1st Link
Bibtex format
@Article{EDI-INF-RR-0375,
author = { Sethu Vijayakumar and Aaron D'Souza and Stefan Schaal },
title = {Incremental Online Learning in High Dimensions},
journal = {Neural Computation},
publisher = {MIT Press},
year = 2005,
month = {Dec},
volume = {17},
pages = {2602-2634},
url = {http://homepages.inf.ed.ac.uk/svijayak/publications/vijayakumar-NeuCom2005.pdf},
}


Home : Publications : Report 

Please mail <reports@inf.ed.ac.uk> with any changes or corrections.
Unless explicitly stated otherwise, all material is copyright The University of Edinburgh