This Meta September 24 2025 paper provides an extensive overview of **Code World Model (CWM)**, a 32-billion-parameter dense decoder-only Transformer designed for **coding and reasoning tasks**, highlighting its architecture and multi-stage training process. Training involves **pre-training, mid-training, and post-training stages** which include supervised fine-tuning (SFT) and joint reinforcement learning (RL) across environments like software engineering (SWE) tasks, coding problems, and mathematics. A core feature is CWM's ability to process and predict **Python execution traces** and perform **agentic interactions** using a minimal set of tools within containerized environments. The document details the model's competitive performance against other large language models on benchmarks like SWE-bench Verified and discusses infrastructure choices, such as **asynchronous RL** and **fp8 matrix multiplication**, used to achieve training efficiency.
Sources:
https://ai.meta.com/temp/research/publications/cwm-an-open-weights-llm-for-research-on-code-generation-with-world-models/
https://scontent-lax3-2.xx.fbcdn.net/v/t39.2365-6/553592426_661450129912484_4072750821656455102_n.pdf?_nc_cat=103&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=0-g1m0kIX7cQ7kNvwFJRCs6&_nc_oc=AdnQYyoahmWWXKZWyQuh4F09IuhBGd08Uwox14N8BdY_tMilZ5_Tl5u7P82HLIJ9RSc8nDy188xiuxmmByXhkJ1S&_nc_zt=14&_nc_ht=scontent-lax3-2.xx&_nc_gid=RcYYIy-y9eIernCj-naXRQ&oh=00_AfYC3ol0MNn_PokD4H3hoOcpxg6OoOsgWfD4pDyThszIpA&oe=68DD28B5
