[SERVER-22599] inMemory storage engine crash when out of memory Created: 12/Feb/16  Updated: 05/Dec/16  Resolved: 19/Apr/16

Status: Closed
Project: Core Server
Component/s: Storage, WiredTiger
Affects Version/s: 3.2.1
Fix Version/s: 3.2.6, 3.3.5

Type: Bug Priority: Major - P3
Reporter: Asya Kamsky Assignee: Geert Bosch
Resolution: Done Votes: 0
Labels: code-and-test
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
is documented by DOCS-9500 Docs for SERVER-22599: inMemory stora... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Completed:
Sprint: Integration 12 (04/04/16), Integration 13 (04/22/16)
Participants:

 Description   

When cache was exceeded, after several "cache full" exceptions mongod crashed with invariant failure.



 Comments   
Comment by Githook User [ 20/Apr/16 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-22599 Test memory full behavior for inMemory storage engine
Branch: v3.2
https://github.com/mongodb/mongo/commit/964aaefeeaf42933b649dc4c1d6c193da063575d

Comment by Githook User [ 20/Apr/16 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-22599 uassert on cache full error
Branch: v3.2
https://github.com/mongodb/mongo/commit/43270827eba78e3a929c54a6501332d3efb289c2

Comment by Daniel Pasette (Inactive) [ 19/Apr/16 ]

geert.bosch, go ahead and backport this now.

Comment by Geert Bosch [ 19/Apr/16 ]

Should describe what happens in memory full situations: a write will fail with ExceededMemoryLimit error (code 146). Later writes may still succeed as old transactions close and/or storage gets reconciled by background threads. However, the database is in degraded mode, and further write attempts should only remove data, and/or indexes.

Comment by Githook User [ 19/Apr/16 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-22599 Test memory full behavior for inMemory storage engine
Branch: master
https://github.com/mongodb/mongo/commit/af65f9793a212fdeffeeebed3a76b11921e53032

Comment by Daniel Pasette (Inactive) [ 06/Apr/16 ]

As discussed yesterday, Geert is planning to add a test for this in a separate inMemory suite using a small cache size using --inMemoryStorageEngineConfigString='cache=128M' or similar.

Comment by Githook User [ 05/Apr/16 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-22599 uassert on cache full error
Branch: master
https://github.com/mongodb/mongo/commit/5d1262cc394d685b59ae3185d7315227085e897d

Comment by Asya Kamsky [ 12/Feb/16 ]

I believe diagnostic data directory may not have been there - I don't think it's related to the failure as I found diag error in other log files, but this was the only case of this crash.

Comment by Daniel Pasette (Inactive) [ 12/Feb/16 ]

Not necessarily related to the failure, but does the directory at /disk1/db/loads_2/ exist is there a reason mongod wouldn't be able to create diagnostic.data?

Generated at Thu Feb 08 04:00:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.