Universal Sequential Decisions in Unknown Environments

Author:	Marcus Hutter (2001)
Comments:	2 LaTeX two-column pages
Subj-class:	Artificial Intelligence; Learning;
ACM-class:	I.2; I.2.6; I.2.8; F.1.3; F.2
Reference:	Proceedings of the 5th European Workshop on Reinforcement Learning (EWRL-5) 25-26, Onderwijsinsituut CKI, (picture gallery)
Remark:	This is a 2 page summary of 60 page technical report
Paper:	LaTeX (17kb)  -  PostScript (87kb)  -  PDF (91kb)  -  Html/Gif
Slides:	PostScript - PDF

Keywords: Artificial intelligence, Rational agents, sequential decision theory, universal Solomonoff induction, algorithmic probability, reinforcement learning, computational complexity, Kolmogorov complexity.

Abstract: We give a brief introduction to the AIXI model, which unifies and overcomes the limitations of sequential decision theory and universal Solomonoff induction. While the former theory is suited for active agents in known environments, the latter is suited for passive prediction of unknown environments.

Table of Contents

• Introduction
• Universal probability distribution
• Bayesian decisions
• More active systems
• Agents in known probabilistic environments
• Sequential decision theory
• Bellman equations
• Reinforcement learning for unknown environment
• Unknown loss function
• The universal AIXI model
• Universally optimal AI systems

BibTeX Entry

@Article{Hutter:01decision,
  author =       "Marcus Hutter",
  title =        "Universal Sequential Decisions in Unknown Environments",
  year =         "2001",
  pages =        "25--26",
  address =      "Manno(Lugano), CH",
  journal =      "Proceedings of the 5th European Workshop on Reinforcement Learning (EWRL-5)",
  number =       "27",
  editor =       "Marco A. Wiering",
  publisher =    "Onderwijsinsituut CKI - Utrecht University",
  series =       "Cognitieve Kunstmatige Intelligentie",
  ISBN =         "90-393-2874-9",
  ISSN =         "1389-5184",
  keywords =     "Artificial intelligence, Rational agents,
                  sequential decision theory, universal Solomonoff induction,
                  algorithmic probability, reinforcement learning, computational
                  complexity, Kolmogorov complexity.",
  url =          "http://www.hutter1.net/ai/pdecision.htm",
  categories =   "I.2.   [Artificial Intelligence],
                  I.2.6. [Learning],
                  I.2.8. [Problem Solving, Control Methods and Search],
                  F.1.3. [Complexity Classes],
                  F.2.   [Analysis of Algorithms and Problem Complexity]",
  abstract =     "We give a brief introduction to the AIXI model, which unifies and
                  overcomes the limitations of sequential decision theory and
                  universal Solomonoff induction. While the former theory is suited
                  for active agents in known environments, the latter is suited for
                  passive prediction of unknown environments.",
}