• Citer

Synthesizing POMDP Policies: Sampling Meets Model-Checking via Learning

par Chakraborty, Debraj ;Majumdar, Anirban ;Mathew, Prince ;Mukherjee, Sayan ;Raskin, Jean-François
Référence Lecture notes in computer science, 16683 LNCS, page (380-402)
Publication Publié, 2026

Article révisé par les pairs

• Accès en ligne
• Détails
• Contenu
• Statistiques

Résumé :

Partially Observable Markov Decision Processes (POMDPs) are the standard framework for decision-making under uncertainty. While sampling-based methods scale well, they lack formal correctness guarantees, making them unsuitable for safety-critical applications. Conversely, formal synthesis techniques provide correctness-by-construction but often struggle with scalability, as general POMDP synthesis is undecidable. To bridge this gap, we propose a synthesis framework that integrates sampling, automata learning, and model-checking. Inspired by Angluin’s L∗ algorithm, our approach utilizes sampling as a membership oracle and model-checking as an equivalence oracle. This enables the synthesis of finite-state controllers with formal guarantees, provided the sampling-induced policy is regular. We establish a relative completeness result for this framework. Experimental results from our prototypical implementation demonstrate that this method successfully solves threshold-safety problems that remain challenging for existing formal synthesis tools. We believe our algorithm serves as a valuable component in a portfolio approach to tackling the inherent difficulty of POMDP synthesis problems.