fangvv/VN-MADDPG

Code for paper "基于多智能体深度强化学习的车联网通信资源分配优化"

What it solves

This project addresses the problem of spectrum sharing and resource allocation in the Internet of Connected Vehicles (IoCV). Specifically, it aims to optimize the transmit power and resource block (RB) selection for Vehicle-to-Vehicle (V2V) links to improve transmission rates and total capacity without compromising Vehicle-to-Infrastructure (V2I) links, especially in high-mobility urban environments where rapid channel changes make centralized management impractical.

How it works

The project implements a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm based on the Centralized Training with Decentralized Execution (CTDE) framework.

  • Agents: Each V2V link is treated as an agent.
  • Observation: Agents observe local states, including fast and large-scale fading across resource blocks, interference levels, and payload status.
  • Action: Agents output continuous values to select a resource block and determine the transmit power level.
  • Reward: A shared reward is calculated based on the total V2I link capacity and V2V transmission success rate.
  • Training: During training, a centralized Critic network aggregates actions from all agents to stabilize learning. At execution time, only the decentralized Actor networks are used, allowing vehicles to make decisions based on local observations.

Who it’s for

Researchers and engineers working on vehicular networks, wireless communication resource allocation, and the application of multi-agent reinforcement learning (MARL) to network optimization.

Highlights

  • CTDE Framework: Uses centralized training to handle the non-stationarity of multi-agent environments while maintaining decentralized execution.
  • Comprehensive Baselines: Includes implementations of MADQN (discrete action space), Single-Agent DDPG (independent learning), and Random policies for performance comparison.
  • Realistic Simulation: Built on the 3GPP TR 36.885 urban grid model with Manhattan layout, simulating LOS/NLOS path loss and shadow fading.
  • Advanced Replay: Implements both uniform and Prioritized Experience Replay (PER) using segment trees for efficient sampling.

Related

  • Dispatch
    OpenAI MADDPG: Learning to Cooperate, Compete, and Communicate

    OpenAI introduced MADDPG, a reinforcement learning algorithm that uses centralized learning and decentralized execution to enable multiple agents to collaborate and compete in complex environments.

  • Project
    fangvv/UAV-DDPG

    A reinforcement learning framework using DDPG to minimize processing delays in UAV-assisted mobile edge computing by optimizing flight paths and task offloading.

  • Project
    AgentR1/Agent-R1

    A modular framework for Agentic Reinforcement Learning that trains multi-step LLM agents by treating each interaction turn as a step-level MDP transition.

  • Project
    TsinghuaC3I/MARTI

    An open-source framework for training LLM-based multi-agent systems using reinforcement learning, featuring tree search augmentation for complex reasoning and code generation.