Swapping a Go heap

The problem:

• a cgroup running two processes with memory.max set to 900 MiB.
• The main process is a Go program with a baseline RSS of around 32 MiB.
• The other, a neighboring HTTP server, uses ~400 MiB (without spikes).
• The Go process spikes to 500+ MiB on io.ReadAll and proto.Unmarshal, which triggers an OOM kill.

I had an idea: what if I set memory.high to 700 MiB and use a swap backend so the kernel can push anonymous pages during spikes? Worth trying. But since I’m running two processes in the same cgroup, how do I know the kernel is going to evict the Go’s heap and not the neighbor’s process heap?

I ran a mock allocator with two setups, both on kernel 6.10 and MGLRU enabled. In the first one, the neighbor’s heap was allocated and not touched for the entire duration of the test. In the second, I forced a read from the heap every 15 seconds.

The two runs behave very differently. In the first run the neighbor’s heap is agressively swapped out, while Go’s pages remain intact until the swap near the end.

In the second run, both heaps are swapped out - since the neighbor’s heap is being read every 15 seconds, we have cycles where Go’s heap is the oldest. For example:

• at t0, the neighor’s heap is read, making its pages young.
• from t0 to t1, if no Go pages are acessed, they become the oldest ones in the cgroup.
• at t1, another neighbor heap read makes those pages young again, leaving Go’s pages as the oldest.

The same cycle then repeats. Major page faults reached a peak of 20,000+ per second in the second run.

What’s the reason behind this? The kernel needs to evict pages, and the way it does that is by checking their age:

 * The aging needs to
 * check the accessed bit at least twice before handing this page over to the
 * eviction. The first check takes care of the accessed bit set on the initial
 * fault; the second check makes sure this page hasn't been used since then.

Go uses a tracing collector, which means that whenever the garbage collector runs, it needs to read everything in the heap that has pointers. In practice, Go’s allocator creates spans and mark spans as scan or noscan. It’s a hint to the garbage collector. If a scan has internal pointers to be walked through, the GC needs to read them all to check what is still alive.

In addition, I checked, for the same 2 runs, what were the % of time the threads got stalled, and the swap I/O MIB/s.

The difference in swapped mebibytes in both cases is… interesting? 4.1 MiB to 2.1 GiB is a big difference. Swap out means the kernel is writing things on disk in the background. Swap in has a different impact on production: it means a thread is waiting until the kernel brings back those pages from disk. In the first run, the kernel wrote 418 MiB and read only 4. In the second, it wrote 2,6 GiB and read 2,1 GiB. This is also reflected in the time threads got stalled on both runs: 0,5s in the first one, and 3,5s in the second.