Context Language Models Outperform State-of-the-Art in Efficiency and Accuracy
Researchers introduced Context Language Models (CLMs), which manage their own context by treating it as a file that the model can update freely. This approach allows the model to learn what context is most important to retain, improving performance in multi-agent systems. CLMs built from existing models surpassed state-of-the-art context management strategies in accuracy and efficiency across various tasks, including a 11.4% accuracy boost with 21.5% fewer FLOPs on BrowseComp-Plus and a 65% improvement in a 24-hour multi-repository task with the same compute. By shifting context management from external control to intrinsic model behavior, CLMs enable both in-context and parametric learning of context-management strategies. The models can be steered with natural-language instructions evolved through a skill-optimization loop, improving accuracy by up to 35.9 points on a context-management task while reducing compute. An online reinforcement learning method improved Qwen3.5-9B performance by 47.6% with 12% fewer FLOPs. Additionally, a co-designed Suffix Cache Reuse reduced server-side compute by 35% relative to standard SGLang at matched performance.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.