back
193 comments
It's important to remember the biggest reason why overcommit exists on Linux and macOS: fork(). When a process forks, the vast majority of the child process's memory is safely shared with the parent process, due to copy-on-write. But a strict accounting would say that the total memory usage of the system has doubled, which is too conservative in most cases. Since forking is so common on Unix, overcommit ends up being necessary pretty quickly. Otherwise, fork/exec would stop working in any process using more than half of the system memory.
wouldn't you just account the COW pages against the parent until they are copied?

kicking the can down the road means there isn't any longer a reasonable correction (failing the allocation), but instead we get to drive around randomly trying to find something to kill.

this is particularly annoying if you are running a service. there is no hope for it to recover - for example by flushing a cache. instead the OS looks around - sees this fat process just sitting there, and .. good news, we have plenty of memory now.

The biggest reason overcommit exists is because it allows the system to operate more efficiently. The reality is most applications touch only some of the pages they allocate, and it's silly for the system to fail a malloc. Often times other expensive cleanup activities can be deferred (you don't really want to drop a handy directory entry cache just so an app can be sure it got physically backed memory for its request, 99.99% of the time).

IIUC Linux was really the first OS to make overcommit so prominent. Most systems were a lot more conservative.

> Otherwise, fork/exec would stop working in any process using more than half of the system memory.

Somehow Solaris manages just fine.

And don't forget that swap memory exists. Ironically, using overcommit without swap is asking for trouble on Linux. Overcommit or no overcommit, the Linux VM and page buffer systems are designed with the expectation of swap.

Most processes that fork know that they are going to fork. Therefore they can pre-fork very early, so as to commit the memory when they only have a bare minimum allocated. Your typical daemon does this anyway.

Some other type of process like an interpreter that can subshell out doesn't know how big the allocation is going to get, would have to pre-fork early on.

In this way, you wouldn't "need" overcommit and the Linux horror of OOM. Well, perhaps you don't need it so badly. Programs that use sparse arrays without mmap() probably need overcommit or lots of swap.

Even the linux docs for it suggest to turn it on if you're working with sparse arrays, there's no mention of fork()

https://www.kernel.org/doc/Documentation/vm/overcommit-accou...

I first learned about this while trying to understand processes being killed by OOM in production. We had python 2.x batch jobs being executed by long-running python worker processes -- some of the arbitrary application code in some of the arbitrary batch jobs would occasionally want to execute some command line tool in a new process, and to create the new process under the hood python's subprocess library would fork/exec, and if the parent worker process had already accumulated a large virtual memory footprint, linux's approximate memory accounting heuristics would kick in during the attempted "fork", decide that we were obviously going to run out of physical memory, and kill the process.

We didn't actually want to fork anything and share gigabytes of virtual memory with the child process, we wanted to spawn an almost entirely independent process to do something and report results, but that got implemented under the hood by fork.

Spawning processes is one area where Windows is more elegant than linux: windows offers spawn. Apparently macos and solaris implement a posix_spawn that avoid the complications of fork/exec.

linux offers posix_spawn, apparently which may may or may not call fork under the hood depending on which libc you're using. If libc implements posix_spawn by calling fork then you're back in the same mess with linux heuristic memory accounting and overcommit. E.g. old versions of glibc will fork when you posix_spawn, newer versions of glibc may vfork . musl apparently will always vfork.

It looks like cpython's subprocess.Popen was patched in python 3.8 to detect some cases where posix_spawn can be used -- it reads as if it will only kick in on linux if it detects a sufficiently new version of glibc: https://github.com/python/cpython/blob/main/Lib/subprocess.p...

edit: Python 3.10 now supports using vfork for linux inside subprocess: https://bugs.python.org/issue35823

  docker run --rm -it --entrypoint=/bin/sh python:3.9-alpine
  # apk add strace
  # strace python -c "import subprocess; subprocess.run(['ls', '-l'])" 2>&1 >/dev/null | grep fork
  fork()                                  = 88

  docker run --rm -it --entrypoint=/bin/sh python:3.10-alpine
  # apk add strace
  # strace python -c "import subprocess; subprocess.run(['ls', '-l'])" 2>&1 >/dev/null | grep fork
  vfork()                                 = 15
edit 2: here's a similar tale from go, replacing use of fork in fork/exec:

https://github.com/golang/go/issues/5838

https://go-review.googlesource.com/c/go/+/37439/

https://about.gitlab.com/blog/2018/01/23/how-a-fix-in-go-19-...

I once complained about malloc happily allocating memory that the physical memory system couldn't satisfy (never let your hand write a check your ass can't cash?) but the more experienced programmer asked me if I'd heard of fractional reserve banking, and if not, whether it bothered me too.
> I once complained about malloc happily allocating memory that the physical memory system couldn't satisfy (never let your hand write a check your ass can't cash?) but the more experienced programmer asked me if I'd heard of fractional reserve banking, and if not, whether it bothered me too.

What if you are worried about both? ;-)

Swap space is the Federal Reserve of memory allocation.
Oh. but is there a scenario where it might be useful to check if a certain amount of memory can be available? Like say you know a certain process uses 6 Gigs for memory but will take a while to get to that point.. and then fail, is it not safe to just error out earlier?
Not the same thing.

malloc() can tell everybody it has the memory but when push comes to shove the OS will have to admit overbooking.

If you're smart, the answer is yes, over reliance on statistical multiplexing scares the shit out of you, because it's all fun and games till the check is due.
This has nothing to do with C. The problem/feature is with the systemcall that does the allocation, and any language has to use it.
Is your issue that the system call doesn't return enough diagnostic information? If so, how would you have done it differently? I'm asking out of curiosity, not out of a reflexive instinct to defend C (which has many problems).
That’s one of the many reasons I think Windows NT kernel is generally better than Linux.

Windows doesn’t do that. When you don’t have enough memory and not enough page file space either, these allocation functions usually do fail returning nullptr.

But this feature is also why malloc on Windows can take a long time to return - seconds or more if you malloc too much. I recently bumped into this problem on the GPU. Since GPU memory in the Windows Display Driver Model requires it to be backed by the host machine’s virtual memory, a malloc on the GPU when running low on space can end up searching & even defragging virtual memory to satisfy the request. Crazy! Just asking for RAM can cause you to land in heavy disk swap. I’ve seen cases of cudaMalloc taking minutes because of how Windows handles allocation, and the same is true of CPU mallocs.
I've found overcommit useful for some scientific computing applications, but you're right it's not the most intuitive default. You can disable memory overcommit on Linux for the same behavior as Windows, if you want.

https://www.kernel.org/doc/Documentation/vm/overcommit-accou...

Also, I found this post that suggests Windows technically does overcommit memory, but only for stacks(‽): https://superuser.com/questions/1194263/will-microsoft-windo...

If you really want that you can just tell Linux not to overcommit. I can foresee your next question though, "Wait, why did I run out of memory? Stupid Linux, I have plenty of RAM".
On Linux, people should be pointed to

https://www.kernel.org/doc/Documentation/vm/overcommit-accou...

Nope. Works fine on Fedora 34:

# free -h

               total        used        free      shared  buff/cache   available
Mem: 31Gi 3.1Gi 2.2Gi 27Mi 25Gi 27Gi

Swap: 15Gi 62Mi 15Gi

# uname -a

Linux athena 5.13.19-200.fc34.x86_64 #1 SMP Sat Sep 18 16:32:24 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

# gcc -o memaloc memaloc.c

# ./memaloc

error!

I'm a little disappointed that the article didn't answer the question, or at least try to. A discussion of using read/write vs mincore vs trying to catch a SIGSEGV would've been a nice addition.
This difference between "malloc() succeeded and physical/swap memory is actually available" also has somewhat corresponding impact on how you measure memory usage.

One approach is RSS, the memory in physical RAM... but what if you're swapping? Then again, maybe you swapped out memory you don't actually need and ignoring swap is fine.

The other approach is "how much memory you allocated", and then you hit fun issues mentioned in this article, like "the OS doesn't actually _really_ allocate until you touch the page".

(Longer version: https://pythonspeed.com/articles/measuring-memory-python/)

IMO strict memory accounting misses the point. It can ensure that all allocated pages fits to VM (or return error during allocation), but more pragmatic memory constraint is whether working set of pages (including code/mmap-based pages) fits in physical RAM. If that is not satisfied, system/application crawls to a halt due to page thrashing and some kind of OOM killer is needed. And that may happen even if strict memory accounting is satisfied.
It’s likely being OOM killed on Linux. Not sure what’s happening on Mac. Try allocating a terabyte of swap and it should run.

Alternatively use mmap & mlock to verify the allocation succeeded, but the process can still be OOM killed at any time for any reason.

If you care, you mmap it, and request MAP_POPULATE.

It would not be very surprising if, once enough people come to depend on this, they break that, too.

Is this really an issue with C? Isn’t it ultimately an OS config issue?
Try that on IAR EWARM on a Cortex-M0 target.

It will absolutely fail (first because size_t is only 32b).

What's behind malloc() is what matters.

That being said, I never would have expected that code to ever succeed! Shows how much I take memory allocation for granted on more sophisticated systems. I can't remember the last time I malloc'd more than a few megabytes.

> If you use a system-level tool that reports the memory usage of your processes, you should look at the real memory usage.

Things are a bit more complicated than that. Because RSS will contain memory mapped to your process that could also be mapped by other processes. That is the sum of RSS on your machine is also higher than your physical memory.

That includes libraries dynamically linked to your executable, but more importantly shared memory mmapped to your process.

A more "fair" estimate exists in the form of PSS (or USS), that will list all mapped regions from all process, and account each process a proportional share of the region.

e.g. If 2 processes mmap `/dev/shm/foo` of 1GB, both will inherit 500GB by PSS computation.

if you want to allocate physical memory, allocate address space with mmap() (malloc with options) and use mlock() to wire the physical pages to the addresses in your process. mlock() will fail if there's not enough physical ram to satisfy your request.
Life is hard for C coders.

Kubernetes made writing poor code a breeze. At work we have microservices crashing 20 times a week but SLOs are not affected since traffic is routed to surviving pods. So we can concentrate on churning features fast instead of writing good code.

This made be curious to check the real and virtual memory of some processes on my laptop (MacBook Air M1).

The real memory size of Safari is ~160MB but virtual memory size is 392GB which doesn't look right. I checked other processes and all the processes have similar virtual memory size which is around ~390GB.

I wonder if this is a bug in Activity Monitor or the virtual memory allocations really are this big for each process.

Overcommit should not be necessary in order to just reserve a range of VM without committing it. Windows gets this right (requiring explicit commit to use memory if you explicitly reserved it earlier). Demand paging on Linux is super-convenient (and my designs exploit it heavily), but you can’t really write robust programs that way.
One of my interview questions starts with "Can a program allocate more memory than is physically available on the server?"

Everybody gets this wrong (which is funny for a binary question) but it starts an interesting discussion through which I hope to learn how much they know about OS and virtual memory.

I thought this was going to be about the more fun point that malloc is allowed to return NULL even on successful allocation (because malloc(0) is allowed to return NULL). This is a quick way to check if the authors of malloc wrappers know what they are doing.
If allocation didn't succeed, you have bigger problems. I personally dislike articles like this one. It is apparent they have never run into the problem in a production scenario.
Ever since I learnt about overcommit I've enjoyed blowing C programmers' minds with it. It's really surprising how many still think malloc fails!
Could this be solved by having a process level attribute that forces for all allocations what mlock(2) does? Does mlock(2) even do this?
You can probably mitigate this with malloc_s or similar that writes 0 bytes to the entire allocated space before returning.
This not "In C", this is a system-related issue. I always disable overcommit as I see no benefit on my systems.
Nope. Works fine on Fedora 34:<BR> # free -h total used free shared buff/cache available Mem: 31Gi 3.1Gi 2.2Gi 27Mi 25Gi 27Gi Swap: 15Gi 62Mi 15Gi # uname -a Linux athena 5.13.19-200.fc34.x86_64 #1 SMP Sat Sep 18 16:32:24 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux # gcc -o memaloc memaloc.c # ./memaloc error!
This is why some embedded systems don't do malloc (IIRC...been a while since I've read much about those).
I tried this program under Ubuntu 20.04 (gcc 9.3.0) and got the "Cannot allocate memory" message.
I suspect calloc() doesn't have this problem as it initializes the area with zeroes.
Is the title a reference to Terry Riley's In C?
Why use malloc() when mmap() does the job
TLDR: "You don't." (Because malloc hands you virtual memory and actually trying to use it might reveal that the system doesn't have the real memory to handle your request.

I kept reading hoping that there was going to be a solution, but not really; there are comments discussing disabling overcommit, but even that's a tradeoff (it does fix this failure mode, but you might not want to actually run a system like that).