RC RANDOM CHAOS

A binary that hands kernel hooks to anyone

zeroserve uses eBPF as a web server's configuration, JIT-compiled and sandboxed in userspace with no kernel BPF subsystem or CAP_BPF involved.

· 5 min read
A binary that hands kernel hooks to anyone

Drop a .c file under .zeroserve/scripts/ and zeroserve compiles it to eBPF bytecode at pack time, with clang and llc. That bytecode runs on every HTTP request. It never enters the kernel. zeroserve loads it into a userspace runtime called async-ebpf, which vendors uBPF, JIT-compiles it to native x86-64, and runs it inside its own ordinary unprivileged process. The kernel’s BPF subsystem and CAP_BPF stay out of it entirely.

zeroserve is a single binary that serves a website from one tar file over HTTP/2 and TLS 1.3. There is no config file. The eBPF program is the configuration: a chain of scripts, run in sorted filename order, that sees every request and decides routing, headers, auth, rate limiting, and proxying. The scripts share a per-request metadata map, and a call to zs_respond or zs_reverse_proxy short-circuits the chain. Keys the scripts write under zs.response.header.* become response headers on everything the server emits, and other keys feed a small template pass that substitutes <zs-meta> placeholders in HTML on the way out.

What’s borrowed here is the eBPF instruction set and the clang toolchain, not the tracing-and-networking machinery most people mean by the name. You write C, clang targets the eBPF backend, and uBPF JITs the result. The kernel pieces that usually come with eBPF, the verifier, the maps, the hook points, and the capability model, are all absent. zeroserve rebuilds its own equivalents in userspace, and the security question worth asking is how well those substitutes cover what the verifier gave you.

The verifier’s two jobs, split

The kernel’s BPF verifier does two things that matter here. It proves a program only touches memory it is allowed to, and it proves the program terminates, with bounded loops and no unbounded back-edges. zeroserve splits those across two mechanisms. Memory safety comes from a pointer cage: the JIT masks every memory access into the program’s own arena, so a stray read or write stays confined to the script’s memory. Termination is handled at runtime rather than proven in advance. The runtime is fully preemptible, so a timer can interrupt JIT-compiled native code mid-execution and return control to the event loop. A script that spins forever stalls only its own request. Each script also runs under a 256 KB memory cap, gets time-sliced off the executor if it runs long, and can call other scripts through zs_call only to a bounded depth.

This is a different trust model from the kernel’s. The verifier rejects a bad program before it runs; zeroserve accepts the program and contains it while it runs. For a server where you own the scripts, that trade is reasonable. You give up the static guarantee and keep a single-threaded event loop responsive under a misbehaving script.

Containment has a measurable cost. The preemption timer fires every 2 ms by default. Fine granularity throttles a runaway quickly but taxes every well-behaved script too. At the default, the eBPF path trails nginx with LuaJIT on a fully dynamic response, about 32k requests per second against 41k. Raising --preempt-timer-interval-ms to 10 recovers roughly 40% of scripting throughput and turns the result around: tuned eBPF beats nginx Lua by about 50% on middleware that shapes an otherwise-static response, and edges it at 47k against 41k on a fully synthetic JSON response. If you run scripts in production, 10 ms is the setting the author recommends.

One tarball, served in place

The whole site is one tar file. zeroserve indexes it on load into a path-to-byte-range map and serves files by issuing byte-range reads against the tarball itself, every read and write submitted through io_uring via the monoio runtime. Nothing is unpacked to disk, so there is no document root for a stray rule to expose, and a deploy is one atomic file swap followed by a SIGHUP. The reload swaps the site, the scripts, and the TLS material in the same process with no dropped connections.

For something framed as zero-config, the helper surface a script can reach is wide: SHA-256, HMAC-SHA256, base64, hex, getrandom, JSON parsing and building, per-key token-bucket rate limiting whose state survives a hot reload, AWS SigV4 signing and presigned URLs, and a complete OIDC relying-party flow using Authorization Code with PKCE that seals the whole session into an XChaCha20-Poly1305 cookie so the server stays stateless. The transport side is TLS 1.3 only, terminated by BoringSSL, with Encrypted Client Hello so the real SNI never appears in cleartext, SNI certificate selection from a directory, JA4 client fingerprinting exposed to scripts, and a transparent ECH relay that forwards undecryptable handshakes byte-for-byte to an upstream so a protected name hides behind a public one.

Benchmarks, per core

A zeroserve instance is a single-threaded event loop, so the benchmarks are per core by design and the scaling unit is more processes, not more threads. Pinned to one core against nginx 1.26 and Caddy 2.11 over HTTPS on a Ryzen 7 3700X, it serves a 174-byte file about 17% faster than nginx with a tighter tail, and runs about even on a 100 KB file at around 780 MB/s. The large-file parity comes from TLS: every server has to encrypt in userspace, so nginx’s sendfile() zero-copy path goes unused and everyone is bound by the same encrypt-and-write loop.

As a reverse proxy it keeps a pool of up to 128 upstream connections per backend and reuses them. For small responses it leads nginx by about 22%, 26.5k against 21.8k, and Caddy by roughly 3.4x. Large proxied bodies reverse that: nginx’s buffering moves bytes more efficiently and pulls ahead, with zeroserve trailing. Idle, an instance sits around 15 MB PSS, more than nginx’s 6 MB and less than Caddy’s 60 MB, and copies running one-per-core share the binary’s code pages.

If your proxied responses are large, nginx is still the better tool. If they are small and numerous, zeroserve is faster. Run production scripts with the preemption timer at 10 ms.

Share

Keep Reading

Latest on the Wire

Full wire →

New signal daily · RSS

Stay in the loop

New writing delivered when it's ready. No schedule, no spam.