PostgreSQL, strict memory overcommit, and the kernel bug that faked an OOM

Ubicloud makes the case for running PostgreSQL under Linux’s strict overcommit policy (vm.overcommit_memory=2), and the reasoning turns on how Postgres handles failure. Because the postmaster forks a shared-memory-backed backend per connection, the kernel’s OOM killer can terminate a backend mid-write and leave shared buffers in an inconsistent state. Postgres treats any killed child as possible corruption, so it drops every connection and forces crash recovery — meaning one out-of-memory event becomes a full-server outage, potentially a long one if there’s a lot of WAL to replay. Strict overcommit sidesteps this by having the kernel refuse risky allocations up front with ENOMEM, which Postgres handles gracefully: the offending backend cancels its transaction and reports an error while everything else keeps running. The trade is turning rare catastrophic failures into routine early ones, and it works best on machines dedicated to Postgres where committed memory is predictable.

The post also serves as a clear primer on Linux’s three overcommit modes and how CommitLimit is derived from overcommit_ratio/overcommit_kbytes plus swap. Then it recounts where the theory broke: weeks after enabling strict overcommit, some databases started throwing out-of-memory errors despite ample free RAM. The smoking gun was /proc/meminfo reporting 651 GB of Committed_AS on an 8 GB VM, versus a few gigabytes on a healthy peer — the counter was wrong by orders of magnitude.

The investigation points at hugetlb accounting. Ubicloud runs with huge_pages=on and 2 GB of shared_buffers, so each backend maps the same 2 GB hugetlb region into its address space (inflating VSZ but not RSS). That region shouldn’t count toward Committed_AS at all, let alone once per backend, making it the prime suspect for a kernel miscount that a ‘three-character’ patch would fix. The excerpt cuts off mid-VMA-inspection, but the operational lesson is already sharp: strict overcommit is a real safety mechanism for Postgres, and it depends entirely on the kernel counting committed memory correctly.