Séminaire (Organisé par l’Equipe de recherche DI)

Francis MAES

Post-doctorat, Université de KU Leuven

Optimized Look-ahead Trees

Mardi 5 février 2013 à 14h en salle A212

Résumé :

Direct policy search (DPS) and look-ahead tree (LT) policies are two popular techniques for solving difficult sequential decision-making problems. They both are simple to implement, widely applicable without making strong assumptions on the structure of the problem, and capable of producing high performance control policies. However, computationally both of them are, each in their own way, very expensive. DPS can require huge offline resources (effort required to obtain the policy) to first select an appropriate space of parameterized policies that works well for the targeted problem, and then to determine the best values of the parameters via global optimization. LT policies do not require any offline resources ; however, they typically require huge online resources (effort required to calculate the best decision at each step) in order to grow trees of sufficient depth.

This presentation introduces optimized look-ahead trees (OLT), a model-based policy learning scheme that lies at the intersection of DPS and LT. In OLT, the control policy is represented indirectly through an algorithm that at each decision step develops, as in LT using a model of the dynamics, a small look-ahead tree until a prespecified online budget is exhausted. Unlike LT, the development of the tree is not driven by a generic heuristic ; rather, the heuristic is optimized for the target problem and implemented as a parameterized node scoring function learned offline via DPS.


FR SHIC 3272

Collegium UTC/CNRS