Researchers identified an asymmetry in learning compositional thinking in LLM

A group of researchers presented a new study dedicated to the study of compositional thinking in language models using reinforcement learning (RL). The authors analyzed how models combine learned skills to solve complex tasks and discovered a fundamental asymmetry in the learning process.

As part of the study, a framework based on dependency graphs was proposed, which formalizes compositional thinking at three levels of complexity. Experiments were conducted on tasks with data structures that provide deterministic reward calculation. Scientists found that training on decomposed skills does not guarantee successful execution of composite tasks, while training on complex composite tasks allows the model to more effectively handle simple components.

This effect, called the "asymmetry from decomposed to composite", was confirmed under conditions of extrapolation by length, structural shifts in distribution, and transfer of skills to previously unseen tasks. The authors also conducted a pilot study on benchmarks for invoking external tools, confirming that the identified pattern persists in practical AI application scenarios.

The results of the study highlight the limitations of current post-training methods, which often focus on individual skills. Understanding this asymmetry allows developers to adjust training strategies to improve the ability of models to generalize and solve multi-step tasks that require logical coherence. The study is published under the Non-exclusive distribution license.