Fast Non-Parametric Bayesian Inference on Infinite Trees

Author:	Marcus Hutter (2004-2005)
Comments:	8 two-column pages
Subj-class:	Probability Theory; Learning
Reference:	Conference on Artificial Intelligence and Statistics (AISTATS 2005)
Report-no:	IDSIA-24-04 and math.PR/0411515
Paper:	LaTeX  -  PostScript  -  PDF  -  Html/Gif
Slides:	PostScript - PDF
C-Code:	BayesTree.c

Keywords: Bayesian density estimation, exact linear time algorithm, non-parametric inference, adaptive infinite tree, Polya tree, scale invariance.

Abstract: Given i.i.d. data from an unknown distribution, we consider the problem of predicting future items. An adaptive way to estimate the probability density is to recursively subdivide the domain to an appropriate data-dependent granularity. A Bayesian would assign a data-independent prior probability to "subdivide", which leads to a prior over infinite(ly many) trees. We derive an exact, fast, and simple inference algorithm for such a prior, for the data evidence, the predictive distribution, the effective model dimension, and other quantities.

BibTeX Entry

@InProceedings{Hutter:04bayestree,
  author =       "M. Hutter",
  title =        "Fast Non-Parametric {B}ayesian Inference on Infinite Trees",
  booktitle =    "Proc. 10th International Conf. on Artificial Intelligence and Statistics ({AISTATS-2005})",
  editor =       "R. G. Cowell and Z. Ghahramani",
  publisher =    "Society for Artificial Intelligence and Statistics",
  pages =        "144--151",
  year =         "2005",
  http =         "http://www.hutter1.net/ai/bayestree.htm",
  url =          "http://arxiv.org/abs/math.PR/0411515",
  ftp =          "http://www.idsia.ch/idsiareport/IDSIA-24-04.pdf",
  keywords =     "Bayesian density estimation, exact linear time algorithm,
                  non-parametric inference, adaptive infinite tree, Polya tree,
                  scale invariance.",
  abstract =     "Given i.i.d. data from an unknown distribution,
                  we consider the problem of predicting future items.
                  An adaptive way to estimate the probability density
                  is to recursively subdivide the domain to an appropriate
                  data-dependent granularity. A Bayesian would assign a
                  data-independent prior probability to ``subdivide'', which leads
                  to a prior over infinite(ly many) trees. We derive an exact, fast,
                  and simple inference algorithm for such a prior, for the data
                  evidence, the predictive distribution, the effective model
                  dimension, and other quantities.",
  _note =        "Acceptance rate: 57/150 = 38\% for posters and 21/150=14\% talks",
}