Archive for the ‘Partially Observable’ Category

Partially Observable Markov Decision Processes

Sunday, October 16th, 2011

Partially Observable Markov Decision Processes

From the webpage:

This web site is devoted to information on partially observable Markov decision processes.

Choose a sub-topic below::

  • POMDP Tutorial – I made a simplified POMDP tutorial a while back. It is still in a somewhat crude form, but people tell me it has served a useful purpose.
  • POMDP Papers – For research papers on POMDPs, see this page.
  • POMDP Code – In addition to the format and examples, I have C-code for solving POMDPs that is available.
  • POMDP Examples – From other literature sources and our own work, we have accumulated a bunch of POMDP examples.
  • POMDP Talks – Miscellaneous material for POMDP talks.


Well, the site has not been undated since 2009.

But, given the timeless nature of the WWW, it shows up just after the Wikipedia page entry on “Partially Observable Markov Decision Processes.” That is to say it was #2 on the list of relevant resources.

Could be that no one has been talking about POMDPs for the last two years. Except that a quick search at Citeseer shows 18 papers there with POMDP in the text.

I understand interests changing, etc. but we need to develop ways to evaluate resources for the timely nature of their data and perhaps just as importantly, to be able to keep such resources updated.

Both of those are very open issues and I am interested in any suggestions for how to approach them.

POMDPs for Dummies

Sunday, October 16th, 2011

POMDPs for Dummies: partially observable Markov decision processes (POMDPs)

From the webpage:

This is a tutorial aimed at trying to build up the intuition behind solution procedures for partially observable Markov decision processes (POMDPs). It sacrifices completeness for clarity. It tries to present the main problems geometrically, rather than with a series of formulas. In fact, we avoid the actual formulas altogether, try to keep notation to a minimum and rely on pictures to build up the intuition.

I just found this today and even with pictures it is slow going. But, I thought you might appreciate something “different” for the week. Something to read, think about, then reread.

If you are taking the Stanford AI course you may remember the mentioning of “partially observable” in week 1. There was a promise of further treatment later in the course.