fangvv/VN-MADDPG

Code for paper "基于多智能体深度强化学习的车联网通信资源分配优化"

What it solves

This project addresses the problem of spectrum sharing and resource allocation in the Internet of Connected Vehicles (IoCV). Specifically, it aims to optimize the transmission rates of Vehicle-to-Vehicle (V2V) links and the total capacity of Vehicle-to-Infrastructure (V2I) links in environments with high vehicle mobility, where centralized management of network resources is impractical due to rapid channel variations.

How it works

The project implements a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm based on the Centralized Training with Decentralized Execution (CTDE) framework.

  • Agents: Each V2V link is treated as an agent. Agents observe their local state (including channel fading, interference, and payload status) and select a resource block (RB) and a transmit power level.
  • Centralized Training: During training, the Critic network for each agent aggregates the actions of all other agents to compute Q-values, allowing the agent to learn better power control strategies.
  • Decentralized Execution: At test time, the Actor network allows each agent to make decisions based solely on its local observations without needing global information.
  • Environment: The system is simulated using a 3GPP urban grid model with Manhattan layout, simulating path loss, shadow fading, and fast fading for both V2V and V2I links.

Who it’s for

Researchers and engineers working on wireless communication, vehicle-to-everything (V2X) communication, and the application of multi-agent reinforcement learning (MARL) for network resource management.

Highlights

  • CTDE Framework: Uses MADDPG to balance centralized learning and decentralized operation.
  • Continuous Action Space: Handles continuous power control and resource block selection.
  • Comprehensive Baselines: Includes implementations of Multi-Agent DQN (MADQN), Single-Agent DDPG (SAMADDPG), and random policies for performance comparison.
  • Prioritized Experience Replay: Implements a segment-tree based prioritized replay buffer to improve training efficiency.

Related

  • Project
    fangvv/UAV-DDPG

    A reinforcement learning framework using DDPG to minimize processing delays in UAV-assisted mobile edge computing by optimizing task offloading and drone flight paths.

  • Dispatch
    OpenAI MADDPG: Learning to Cooperate, Compete, and Communicate

    OpenAI introduced MADDPG, a reinforcement learning algorithm that uses centralized learning and decentralized execution to enable multiple agents to collaborate and compete in complex environments.

  • Project
    AgentR1/Agent-R1

    A modular framework for Agentic Reinforcement Learning that trains multi-step LLM agents by treating each interaction turn as a step-level MDP transition.

  • Project
    TsinghuaC3I/MARTI

    An open-source framework for training LLM-based multi-agent systems using reinforcement learning, featuring tree search augmentation for complex reasoning and code generation.