Switching Models? Switch Your Harness Too
Researchers benchmarked 22 harness configurations against five language models across 62 terminal tasks to identify optimal cost-accuracy combinations. The best-value harness varies significantly by model: Codex CLI for GLM-5.3, Pi for GLM-5.3 Flash, ZCode for DeepSeek, Oh My Pi for Muse, and Alma for Kimi-K3. Only four harnesses occupy the efficiency frontier, meaning most alternatives deliver poor value across all models.
Harness choice creates meaningful cost variation beyond what model selection alone determines. With a single model fixed, costs swing 1.8 to 5.0 times between the cheapest and most expensive harnesses—a spread verified as statistically significant against measurement noise. Counterintuitively, spending more does not improve accuracy. The highest-passing harness typically ranks middle-tier in efficiency, and across four of five models, the most expensive harness by cost-per-pass actually achieved lower pass rates than the cheapest option.
Practitioners switching models should re-evaluate harness selection. While models drive 92.8% of overall cost variance, harness-model pairing accounts for 4.9% additional variation in deployment costs. For efficiency-focused teams, selecting the right harness for each model can materially reduce expenses without sacrificing performance.
Read the full article
Continue reading at Hacker News →This is an AI-generated summary. Read the original for the full story.