facebookresearch/cwm

Research code artifacts for Code World Model (CWM) including inference tools, reproducibility, and documentation.

What it solves

Code World Model (CWM) is designed to improve how AI models generate code and reason about it. Specifically, it addresses the gap in understanding how specific code changes or system commands actually affect the state of a program or a computer system, allowing for more accurate software engineering tasks.

How it works

CWM is a 32-billion-parameter LLM that has been trained using a specialized approach:

  • Mid-training: The model was trained on a large volume of observation-action trajectories from Python execution traces and agentic interactions within containerized environments.
  • Post-training: It underwent extensive multi-task reinforcement learning (RL) in environments focused on verifiable coding, mathematics, and multi-turn software engineering.

Who it’s for

Researchers and developers who want to advance the state of the art in code generation, neural debugging, and agentic software engineering.

Highlights

  • Open Weights: Provides pre-trained, SFT, and instruction-tuned versions of the model.
  • World Model Capabilities: Trained to reason about the state of a program or system.
  • Neural Debugging: Includes demos showcasing the model's ability to act as a neural debugger.
  • Benchmark Performance: Optimized for key benchmarks like SWE-bench Verified, LiveCodeBench, AIME, and MATH.

Related

  • Dispatch
    The Gap Between Open Weights and Closed Source LLMs

    An analysis of the Artificial Analysis Intelligence Index suggests that while open weights models are rapidly closing the gap in coding capabilities, the overall performance lag behind closed source frontier models remains relatively stable at approximately five months.

  • Dispatch
    Qwen-AgentWorld release: language world model for seven domains and its impact on general agents

    Qwen releases Qwen‑AgentWorld, a language world model that simulates seven agent environments and improves general agents via controllable simulation and unified next‑state prediction.

  • Project
    evidentlyai/evidently

    An open-source Python framework to evaluate, test, and monitor ML and LLM-powered systems, helping developers detect data drift and ensure output quality.

  • Project
    ServiceNow/PipelineRL

    A scalable asynchronous reinforcement learning framework for LLM agents that uses in-flight weight updates to maximize GPU utilization while maintaining on-policy data freshness.

  • Dispatch
    Code Llama Release Notes

    Code Llama is a family of open-access models based on Llama 2, specialized for code tasks with support for infilling and long-context windows up to 100,000 tokens.