Gemma 4 + llama.cpp MTP turns a Mac into a usable offline coding agent at 72 tok/s

Repeated internet outages pushed one developer to build a fully local coding agent on an M1 Max with 64 GB of unified memory, and the resulting walkthrough shows the pieces have finally matured. The stack pairs llama.cpp (built with Metal) serving Gemma 4 26B-A4B in Q4 GGUF format with the Pi terminal agent connected over llama.cpp’s OpenAI-compatible API. The headline gain comes from Multi-Token Prediction: loading Gemma 4’s Q8 MTP head as a speculative draft model lifted generation from 58 to 72.2 tokens per second — roughly a 24% improvement with no prompt-processing penalty. The speculative draft depth proved hardware-dependent; sweeping values from 1 to 6 found 3 optimal on this machine, echoing Unsloth’s advice not to trust defaults.

Two findings stand out. First, llama.cpp outperformed Apple’s own MLX framework on Apple silicon, both with and without MTP — years of cross-platform optimization apparently beating the platform-native option. Second, vision support is a configuration problem as much as a model problem: the 26B variant needs Gemma 4’s multimodal projector loaded server-side, and the Pi model entry had to declare image input before screenshots would flow through, but enabling it cost nothing in text-generation speed.

The practical upshot is that an agent capable of tool calls and reading screenshots of its own output now runs at usable speeds on a laptop-class chip from 2021, entirely offline, in about 17 GB of disk. The post includes the full build, download, and server commands, making it a copy-paste path for anyone wanting a coding agent that survives an ISP failure — or that keeps code off third-party APIs.