Sequence Prediction based on Monotone Complexity
Keywords: Sequence prediction; Algorithmic Information Theory;
Solomonoff's prior; Monotone Kolmogorov Complexity;
Minimal Description Length; Convergence;
Self-Optimizingness.
Abstract: This paper studies sequence prediction based on the
monotone Kolmogorov complexity Km=-log m, i.e. based on
universal MDL. m is extremely close to Solomonoff's prior M,
the latter being an excellent predictor in deterministic as well
as probabilistic environments, where performance is measured in
terms of convergence of posteriors or losses. Despite this
closeness to M, it is difficult to assess the prediction quality
of m, since little is known about the closeness of their
posteriors, which are the important quantities for prediction.
We show that for deterministic computable environments, the
"posterior" and losses of m converge, but rapid convergence
could only be shown on-sequence; the off-sequence behavior is
unclear. In probabilistic environments, neither the posterior nor
the losses converge, in general.
BibTeX Entry
@InProceedings{Hutter:03unimdl,
author = "Marcus Hutter",
title = "Sequence Prediction based on Monotone Complexity",
booktitle = "Proceedings of the 16th Annual Conference on Learning Theory ({COLT-2003})",
series = "Lecture Notes in Artificial Intelligence",
editor = "B. Sch{\"o}lkopf and M. K. Warmuth",
publisher = "Springer",
address = "Berlin",
pages = "506--521",
year = "2003",
http = "http://www.hutter1.net/ai/unimdl.htm",
url = "http://arxiv.org/abs/cs.AI/0306036",
ftp = "ftp://ftp.idsia.ch/pub/techrep/IDSIA-09-03.ps.gz",
keywords = "Sequence prediction; Algorithmic Information Theory;
Solomonoff's prior; Monotone Kolmogorov Complexity;
Minimal Description Length; Convergence;
Self-Optimizingness",
abstract = "This paper studies sequence prediction based on the
monotone Kolmogorov complexity $\Km=-\lb m$, i.e.\ based on
universal MDL. $m$ is extremely close to Solomonoff's prior $M$,
the latter being an excellent predictor in deterministic as well
as probabilistic environments, where performance is measured in
terms of convergence of posteriors or losses. Despite this
closeness to $M$, it is difficult to assess the prediction quality
of $m$, since little is known about the closeness of their
posteriors, which are the important quantities for prediction.
We show that for deterministic computable environments, the
``posterior'' and losses of $m$ converge, but rapid convergence
could only be shown on-sequence; the off-sequence behavior is
unclear. In probabilistic environments, neither the posterior nor
the losses converge, in general.",
}