On Windows 10 and 2012 R2,the parameter dwMaximumWorkingSetSize of SetProcessWorkingSetSizeEx must be >= (dwMinimumWorkingSetSize + 0x7000). The 0x7000 appears to be undocumented and discovered during adhoc testing.

This can be triggered by secure memory exhaustion on Windows. In 4.0, the Encrypted Storage Engine uses more secure memory per database which means it is likely to encounter this.