LLM engineering
41 posts
A 4B model outplans Postgres
A 4B model proposes faster Postgres query plans, but the validation gate and fallback - not the model - are what make the 81% speedup safe to run.
Distillation makes fast followers, never frontier leaders
Distilling frontier models is a real, cheap fast-follow strategy for small labs - but only for verifiable tasks and legally usable teachers.
Google quietly broke the search-scraping stack
Google's 2025 anti-scraping update killed cheap SERP scraping. How to rebuild AI search pipelines on sanctioned APIs, validation, and budget controls.
Turning off training does not protect your data
OpenAI keeps re-enabling 'allow training.' Why a UI toggle is not a data control, and the gateway-and-contract architecture that makes training exposure impossible by design.
SWE-2 is the wrong model for almost everything
SWE-2 is a code-execution node, not a generalist replacement. How to eval it against your own repos and where it actually fits in production pipelines.
The weaker model matters more than the smarter one
DeepSeek v4.1 Flash cuts token cost, not the need for validation - use it in cascades, verification loops, and long-context pipelines that hold up in production.
Context is the whole bill
Portal by Spotify cut Claude Code token usage ~90% by serving scoped code slices instead of whole files, and why that matters for production AI systems.
Same task, different bets
Claude, Codex and Cursor pick different tools across 17k runs - here is how to measure that revealed policy and wrap it in controls before it hits production.
The best terminal assistant has no AI
TERMy runs without an LLM to show where deterministic retrieval beats AI-assisted terminal work: bounded inputs, known answers, instant offline lookups.
Three model calls, finished before you blink
Qwen 3.8 27B at 1500 tokens/s on Cerebras is an orchestration budget, not a quality upgrade - spend it on validation and decomposition, not more agents.
The 150x model tax
Cost, latency, and quality trade against each other in LLM inference. How to route work, use serving-layer levers, and architect systems on the efficient frontier.
A smarter model would have leaked it too.
GitHub's AI agent leaked private repos not from a bug but a design failure. How two-plane architecture, scoped tokens, and deterministic validation stop it.