JuliaPOMDP/POMDPs.jl

What it solves

POMDPs.jl provides a standardized core interface for defining and solving problems involving Markov Decision Processes (MDPs) and Partially Observable Markov Decision Processes (POMDPs). It allows developers to express complex decision-making problems under uncertainty using a common programming vocabulary, making it easier to write solver software and run simulations efficiently.

How it works

The project establishes a core interface that defines how MDPs and POMDPs should be structured. It is supported by a complementary package, POMDPTools, which acts as a standard library providing essential implementations for policies, belief updaters, distributions, and simulators. This ecosystem allows for a modular approach where users can define a problem once and then apply various solvers (available as separate packages) to find optimal or near-optimal policies.

Who it’s for

This toolkit is designed for researchers and developers working on decision-making under uncertainty, reinforcement learning, and robotics, as well as those who need to model systems where the state is not fully observable.

Highlights

• Broad Ecosystem: Integrates with a wide array of MDP and POMDP solvers, ranging from offline value iteration to online Monte Carlo Tree Search (MCTS).
• Cross-Language Support: Can be used with Python via quickpomdps or pyjulia.
• Interoperability: Provides two-way integration with CommonRLInterface and the JuliaReinforcementLearning ecosystem.
• Flexible Modeling: Supports both discrete and continuous states, actions, and observations.