RC RANDOM CHAOS

Redis Creator Unveils Local LLM Inference Engine

· via Hacker News

Original source

From the creator of Redis; run LLM locally with ds4

Hacker News →

DwarfStar 4 (ds4) is a specialized inference engine designed for local deployment of large language models (LLMs) on high-memory Mac, CUDA, and ROCm machines. It supports models like DeepSeek V4, GLM 5.x, and Qwen3.8, offering text and vision capabilities through a unified stack that includes CLI, HTTP APIs, and a native agent. The engine employs asymmetric 2-bit quantization to compress routed experts while preserving critical shared paths, making it practical for local use. Ds4 also features SSD streaming, tensor parallelism, and session batching to optimize performance. The project is open-source under the MIT license and is tailored for specific model families rather than being a generic GGUF runner.

Read the full article

Continue reading at Hacker News →

This is an AI-generated summary. Read the original for the full story.