RC RANDOM CHAOS

Decision Models Beat LLMs for Simple Tasks

· via Hacker News

Original source

Not Everything Needs an LLM

Hacker News →

Decision models return probabilistically-calibrated answers with explicit confidence scores, contrasting with LLM verbosity and unwarranted certainty. PostHog uses them in Replay Vision to filter personal information from session summaries. Two yes/no questions reduce false positives to about one-tenth the rate of a single question; in production, 98.7% of summaries skip expensive LLM processing entirely. By setting thresholds pragmatically—0.4 to catch privacy risks, 0.85+ where false positives cost more—the team avoids unnecessary calls while preventing data leaks.

Decision models also route requests between cheaper and capable models and evaluate simple face-value inputs. PostHog applies them to SQL chart visualization suggestions and to route Slack bot queries to the most cost-effective model. Running 193.6x faster and costing 444.6x less than frontier models, they excel at filtering tasks. However, they fail at multi-step reasoning—an attempt to rank PR review findings achieved only 0.52 AUC. Thresholds vary by model and task, requiring per-project tuning. PostHog developed Jeeves, a hybrid decision model that reasons when uncertain, now their standard for unspecified decision calls.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.