Small open-weights LLM learns to beat Postgres on join-heavy query plans

Query optimizers remain a stubbornly hard problem—join ordering alone is NP-hard, and Postgres can’t count actual row cardinalities during planning, so it leans on statistics and a uniform-distribution assumption that fails badly when data is skewed. That combination leaves real performance on the table, a gap that database researchers have documented over the past decade. This experiment reframes the problem as a reinforcement-learning target: query plans are easy to score because there’s a single objective, execution time, making them well suited to a model that can be rewarded for producing faster plans.

The author post-trained a 4-billion-parameter open-weights model using supervised fine-tuning plus agentic RL, distilling from roughly 500 trajectories of a larger agent to bootstrap competence. The result was a 44.7% latency reduction across 113 join-heavy IMDb-style queries—striking given the base model initially couldn’t even produce a valid plan for 99 of them. Getting there required real systems engineering: a custom GRPO variant to handle noisy reward signals, a measurement rig that suppressed Linux page-cache contention across concurrent Postgres containers, and a split setup running vLLM and the trainer on rented dual-H100 hardware while Postgres instances ran locally.

The significance is less about dethroning Postgres and more about a repeatable pattern: any optimization problem with a cheap, unambiguous verifier is a candidate for small-model RL. Cost-based planners have hand-tuned heuristics precisely because exhaustive search is intractable, and a learned policy that adapts to actual data distributions could complement or eventually augment them. Reproducing the headline gains in production would still mean confronting generalization beyond the benchmark and the overhead of invoking a model in the planning path.