infer-actively/pymdp

What it solves

pymdp provides a framework for simulating agents that use Active Inference, a theoretical approach to intelligence where agents act to minimize surprise (variational free energy). It specifically addresses the challenge of creating agents that can simultaneously learn about their environment (epistemic value) and maximize rewards in discrete state space Markov Decision Process (MDP) environments.

How it works

The library implements the mathematical operations of Active Inference in Python, utilizing NumPy and JAX for efficient computation. It allows users to define an agent's internal model of the world (consisting of observation matrices A, transition matrices B, and preference matrices C) and then simulate the agent's behavior as it interacts with an environment. The agent performs state inference to determine its current state and then infers policies to select actions that minimize expected free energy.

Who it’s for

This tool is designed for researchers and developers interested in cognitive science, computational neuroscience, and AI agents that exhibit curiosity-driven exploration and goal-directed behavior.

Highlights

• Epistemic Drive: Agents naturally seek out information to reduce uncertainty (curiosity) without needing explicit reward shaping.
• JAX-powered: Supports high-performance computations and JAX-first implementation for efficient simulations.
• Validated: Low-level mathematical operations are benchmarked and validated against the SPM MATLAB implementation.
• Discrete State Spaces: Specifically optimized for Active Inference in discrete state spaces.