previous  home  search  LaTeX  -  PostScript  -  PDF  -  Html/Gif   contact    up    next  

Sequence Prediction based on Monotone Complexity


Author: Marcus Hutter (2003)
Comments: 15 pages
Subj-class: Probability Theory; Compexity; Learning
Reference: Proceedings of the 16th Annual Conference on Learning Theory (COLT 2003) pages 298-312
Report-no: IDSIA-09-03 and cs.AI/0306036
Paper: LaTeX  -  PostScript  -  PDF  -  Html/Gif 
Slides: PostScript - PDF

Keywords: Sequence prediction; Algorithmic Information Theory; Solomonoff's prior; Monotone Kolmogorov Complexity; Minimal Description Length; Convergence; Self-Optimizingness.

Abstract: This paper studies sequence prediction based on the monotone Kolmogorov complexity Km=-log m, i.e. based on universal MDL. m is extremely close to Solomonoff's prior M, the latter being an excellent predictor in deterministic as well as probabilistic environments, where performance is measured in terms of convergence of posteriors or losses. Despite this closeness to M, it is difficult to assess the prediction quality of m, since little is known about the closeness of their posteriors, which are the important quantities for prediction. We show that for deterministic computable environments, the "posterior" and losses of m converge, but rapid convergence could only be shown on-sequence; the off-sequence behavior is unclear. In probabilistic environments, neither the posterior nor the losses converge, in general.

 previous  home  search  LaTeX  -  PostScript  -  PDF  -  Html/Gif   contact    up    next  

BibTeX Entry

@InProceedings{Hutter:03unimdl,
  author =       "Marcus Hutter",
  title =        "Sequence Prediction based on Monotone Complexity",
  booktitle =    "Proceedings of the 16th Annual Conference on Learning Theory ({COLT-2003})",
  series =       "Lecture Notes in Artificial Intelligence",
  editor =       "B. Sch{\"o}lkopf and M. K. Warmuth",
  publisher =    "Springer",
  address =      "Berlin",
  pages =        "506--521",
  year =         "2003",
  http =         "http://www.hutter1.net/ai/unimdl.htm",
  url =          "http://arxiv.org/abs/cs.AI/0306036",
  ftp =          "ftp://ftp.idsia.ch/pub/techrep/IDSIA-09-03.ps.gz",
  keywords =     "Sequence prediction; Algorithmic Information Theory;
                  Solomonoff's prior; Monotone Kolmogorov Complexity;
                  Minimal Description Length; Convergence;
                  Self-Optimizingness",
  abstract =     "This paper studies sequence prediction based on the
                  monotone Kolmogorov complexity $\Km=-\lb m$, i.e.\ based on
                  universal MDL. $m$ is extremely close to Solomonoff's prior $M$,
                  the latter being an excellent predictor in deterministic as well
                  as probabilistic environments, where performance is measured in
                  terms of convergence of posteriors or losses. Despite this
                  closeness to $M$, it is difficult to assess the prediction quality
                  of $m$, since little is known about the closeness of their
                  posteriors, which are the important quantities for prediction.
                  We show that for deterministic computable environments, the
                  ``posterior'' and losses of $m$ converge, but rapid convergence
                  could only be shown on-sequence; the off-sequence behavior is
                  unclear. In probabilistic environments, neither the posterior nor
                  the losses converge, in general.",
}
      
 previous  home  search  LaTeX  -  PostScript  -  PDF  -  Html/Gif   contact    up    next