RC RANDOM CHAOS

Higher Reasoning Costs More, Improves AI Code Agents Less

· via Hacker News

Original source

SlopCodeBench 10/2026 – Opus 5.5, Astra wins overall, xhigh considered harmful

Hacker News →

Astra topped SlopCodeBench with a 16.3% strict pass rate on 36 problems with evolving requirements, a modest increase from gpt-5.5’s 14.8% five months earlier. Other frontier models clustered between 12.2-16.3% for strict passes, with less rigorous core metrics hitting 65.8-74.0%. The benchmark’s design—requiring agents to maintain code functionality as requirements change—tests a realistic development scenario more demanding than static coding problems.

Higher reasoning effort proved counterproductive. Testing “xhigh” configurations, which cost 2.11-4.22x more than medium settings, failed to reliably improve strict pass rates. Opus 5.5 actually performed better on medium effort (28 strict passes versus 25 on xhigh). Only Astra benefited from increased reasoning, suggesting expensive inference doesn’t consistently yield better code quality. Code analysis revealed all models left substantial debris—debugging code, temporary files, and test artifacts—with Opus Medium producing the most despite higher core pass rates, indicating a disconnect between code cleanliness and functional correctness.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.