Next week's seminar (see http://www.iro.umontreal.ca/article.php3? id_article=107&lang=en):
Aggregate Markov Decision Processes
by Hasan Mirza, McGill University
Location: McConnell Engineering (McGill), room 103 Time: April 8th 2008, 10am
We consider a special type of Markov decision problem in which an agent maintains an infiitely divisible collection of identical "machines," each described by a standard finite-state MDP. The agent is subject to constraints on fraction of the machines receiving each available action. We model the collection of MDPs as a single MDP by looking at the frequency at which each state is observed in each underlying MDP. Although this MDP has continuous state and action spaces, its transitions are deterministic, and its structure leads to interesting properties such as a convex value function. We present a linear-programming receding-horizon control technique for use under the discountedcost performance criterion, and investigate its behaviour through experiments on example problems. The experimental results show that the technique is viable on practically-sized problems. Finally, we present a probabilistic result that relates the frequency MDP to the model where each machine is treated separately and the collection is not infinitely divisible.