whenever the cache fills, the inMemory storage engine should either start using a LAS file, or it should abort.
Both of those options are difficult with inMemory storage engine.
use the LAS file
The inMemory storage engine doesn't write any content to disk, and can be run on a diskless machine. The lookaside file is the WiredTiger cache overflow mechanism, and involves writing content to disk. i.e: the storage engine will stop being inMemory if we go down this route. Doing this is also quite difficult from an implementation point of view. The lookaside mechanism is paired up with the reconciliation process (i.e: the process of writing content to disk), since inMemory doesn't write content to disk we'd need to make changes to how content makes it into the lookaside file.
it should abort
It's also difficult for the storage engine to determine when to abort. WiredTiger is limited in what it can do by the state of things outside of the storage engine, and so can't know if the application will change something that allows progress to be made. We have a check when running in diagnostic mode that if no progress has been made for 5 minutes the process is aborted, I don't think it's necessarily what end users want. For example if a user has a replica set where a secondary node goes down for 10 minutes, and thus the commit point lags and the cache on the primary becomes full. It's not clear which behavior is preferable to the end user, but I guess that it's generally for the primary to stall until the secondary comes back online - especially with the inMemory storage engine - since bringing a node back online requires a resync.
I feel like I've thrown out a set of problems here without suggesting a solution. If this functionality was user configurable I'd be much happier adding in an "abort if stuck for X minutes" configuration option for WiredTiger - would that be a reasonable path forward?
|