In Windows, when the size of the data files is close to half the virtual address space limit, then the files can be initially opened, mapped and used (collection created and extents allocated) just fine. However, when the server is merely stopped and restarted (where extents have been allocated for a collection), it crashes with an inability to map the files.

I've narrowed this issue down to a change between 2.4.0-rc0 and 2.4.0-rc1, though it still exists in 2.4.11, 2.6 and 2.7 (though it presents slightly differently in 2.6 and 2.7 than it does in 2.4). It looks like some of the data files might be somehow being mapped multiple times? Some of them are certainly unmapped several times. Maybe related to SERVER-12567?

In Windows 2008 R2, the virtual address space limit for 64 bit user processes is 8TB. I've done all of this testing without journalling to simplify things, but when I was previously looking at it with journalling on, the situation was similar but with an effective limit of 4TB instead. The results are the same whether the "2008plus" or "legacy" win32 x64 builds are used.

A workaround is to use Windows 2012 R2 instead of 2008 R2, where the limit is 128TB instead of 8TB. However, this problem will still affect Windows 2012 R2 for datasets around the 32TB mark (with journalling).

By contrast, in Linux if I use "ulimit -v 10485760" to limit the virtual address space to 10GB, then all of these versions have the expected behaviour, ie. they are able to

• create 9GB of data files
• restart and then open the data files.

Very verbose logfiles are attached. They show the results for

• Windows 2008 R2 vs Linux (10GB vmem limit)
• MongoDB versions 2.4.0-rc0, 2.4.0-rc1, 2.6.4, and 2.7.5
• Creating capped collections of various sizes.

The Windows logfiles do not show any file allocation messages. This is because the files were allocated using an external tool that used (the Windows equivalent of) fast_allocate. (Otherwise allocating TBs of data files on Windows takes hours instead of seconds, even on SSDs. Any fast allocation bugs don't matter, since this is only testing the ability to mmap files.) You can tell when the dbpath has been cleared out by when local.ns gets allocated. A useful command to see the main timeline in each log is something like:

grep -E 'create collection test.|MongoDB starting|dbstats|assert|Map.*errno|allocating new datafile .*local.ns' mongod-2.4.0-rc1.log

Some of the smaller tests were done on an i2.8xlarge instance with 8x 800GB local SSDs in RAID0 (~6TB). The tests above this size used a hs1.8xlarge with 16x 2TB local disks in RAID0.

The results of the tests are:

Where failures occur, the messages are: