[Cryptography] Swap space (Re: It's all K&R's fault)

Jerry Leichter leichter at lrw.com
Thu Apr 24 14:42:47 EDT 2014


On Apr 24, 2014, at 1:50 PM, Peter Gutmann <pgut001 at cs.auckland.ac.nz> wrote:
>> Well, Windows does not use fork()+exec(); it uses spawn() and its variants.
>> Hence it avoids the whole vfork() / "memory overcommit" mess.
> 
> Aren't many fork()s now pretty close to vfork(), via COW?
This is a messy area.  On first glance, just after a fork(), the only additional resources needed are those needed for another entry in the process table, and maybe some similar entries (e.g., for file descriptors) - trivial amounts of space.

The reality is more complicated.  Somewhere, there are maps describing the virtual memory of all processes.  Those can be of non-trivial size.  Depending on the architecture and the implementation, there are limits to how much of the *maps* can be shared.  What can't be shared has to be allocated an initialized. (Back in VAX days, the maps for P0/P1 lived in S0 - and machines eventually got to the point where there was no room in S0 to map the desired number of processes!)

Also, COW requires that there be some place to copy *too*.  As previously noted, you can allocate all necessary swap space up front, or overcommit, wait until it's needed - and then somehow deal with the discovery that there isn't enough swap space available.  After a fork(), if you don't overcommit, you need to allocate enough space to cover the possibility that every writable page in the new process actually *is* written.

An interesting effect of growing address spaces and faster CPU's is that the fraction of data pages (relative to code and other pure pages) has been going up for many years.  While programs are much bigger now than they used to be, all programs spend most of their time in loops, and those loops haven't gotten all that more complex - they just now traverse larger and larger datasets.  So your fork'ed process today has many more writable pages for which swap needs to be reserved.

There are all kinds of unexpected side-effects.  Here's an interesting one:  A very large program does a fork to start another, also-large, program, then exits.  (This particular situation is the same on a system with a spawn() function.)  There should be plenty of free space - the large program just released a whole bunch - but there's a timing issue:  The large program may have freed the memory, but it takes time for the OS to "digest" that memory and return it to the free list.  The dirty pages need to be written; anything added to the free list has to be zeroed.  Implementations often deliberately cap the rate at which they will free pages to avoid starving other processes on the system of CPU.  Actually making large numbers of pages available could take seconds a number of years ago when I discovered this effect in tracking down a problem a customer was running into.  I have no idea if it's gotten better, worse, or stayed the same over the intervening years.
                                                        -- Jerry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4813 bytes
Desc: not available
URL: <http://www.metzdowd.com/pipermail/cryptography/attachments/20140424/d9bb98e3/attachment.bin>


More information about the cryptography mailing list