[SERVER-19707] A crash after dropDatabase leaves the server unable to restart Created: 31/Jul/15  Updated: 06/Dec/22  Resolved: 14/Sep/18

Status: Closed
Project: Core Server
Component/s: MMAPv1
Affects Version/s: 3.0.5
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Nelson Elhage Assignee: Backlog - Storage Execution Team
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-8358 "Move chunk commit failed" shutdown l... Closed
related to SERVER-7861 Add marker file for journaling to ens... Closed
Assigned Teams:
Storage Execution
Operating System: ALL
Steps To Reproduce:
  • Start a mongod using MMAPv1
  • create a database and drop it:
    > db.getSiblingDB('victim')['dummy'].insert( {i:0}

    )
    WriteResult(

    { "nInserted" : 1 }

    )
    > db.getSiblingDB('victim').dropDatabase()

    { "dropped" : "victim", "ok" : 1 }
  • `kill -9` the `mongod` process to simulate a crash
  • restart `mongod` on the same dbpath. You will observe:

2015-07-31T22:40:06.247+0000 W - [initandlisten] Detected unclean shutdown - /tmp/mongo3/mongod.lock is not empty.
2015-07-31T22:40:06.252+0000 I STORAGE [initandlisten] **************
old lock file: /tmp/mongo3/mongod.lock. probably means unclean shutdown,
but there are no journal files to recover.
this is likely human error or filesystem corruption.
please make sure that your journal directory is mounted.
found 2 dbs.
see: http://dochub.mongodb.org/core/repair for more information
*************

Participants:

 Description   

dropDatabase syncs and truncates the journal, which can result in there being no journal files on disk.

If the database crashes after dropDatabase and before any subsequent write operations (which would re-create the journal), on following startup, `mongod` will observe a `mongod.lock`, and no journal files, and assume that a non-journalled `mongod` crashed, and that the dbpath needs repair, even though it is in a completely consistent state thanks to the `sync`.

We encounter this periodically on developer machines, since our application test suite drops databases after it runs, and it's not uncommon for a developer's mongod to see no other write traffic outside of tests.



 Comments   
Comment by Daniel Pasette (Inactive) [ 03/Aug/15 ]

Nelson, thanks for the report and detailed repro.

Generated at Thu Feb 08 03:51:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.