This February 2025 paper introduce CodeI/O, a novel training method for Large Language Models (LLMs) that enhances general reasoning abilities by transforming code into an input-output prediction task. Instead of focusing on generating code, CodeI/O trains models to predict inputs or outputs of a given code in natural language Chain-of-Thought (CoT) rationales. This approach allows LLMs to learn universal reasoning primitives embedded in code, such as logic flow and decision-making, while decoupling them from specific programming syntax. An improved version, CodeI/O++, further refines training data through multi-turn revision based on execution feedback. Experimental results demonstrate that both CodeI/O and CodeI/O++ lead to consistent and balanced performance improvements across a wide range of symbolic, scientific, logical, and mathematical reasoning tasks in LLMs.
Source:
https://arxiv.org/html/2502.07316v1
