Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-16666

repairpath used with directoryperdb on mmapv1 may clean up database directory

    • Storage Execution
    • ALL
    • Hide
      sajack@linux-p7yj /home/sajack/mongo git SERVER-16615 () % ./mongod --dbpath db1 --repairpath db1/path --directoryperdb                                                                                                             :)
      2014-12-24T11:51:38.421-0500 I CONTROL  [initandlisten] MongoDB starting : pid=6714 port=27017 dbpath=db1 64-bit host=linux-p7yj.site
      2014-12-24T11:51:38.421-0500 I CONTROL  [initandlisten] _DEBUG build (which is slower)
      2014-12-24T11:51:38.421-0500 I CONTROL  [initandlisten] 
      2014-12-24T11:51:38.421-0500 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
      2014-12-24T11:51:38.421-0500 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
      2014-12-24T11:51:38.421-0500 I CONTROL  [initandlisten] 
      2014-12-24T11:51:38.421-0500 I CONTROL  [initandlisten] db version v2.8.0-rc4-pre-
      2014-12-24T11:51:38.421-0500 I CONTROL  [initandlisten] git version: 6d217f7a83979c441288611a06b08857eda0234c modules: subscription
      2014-12-24T11:51:38.421-0500 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1j 15 Oct 2014
      2014-12-24T11:51:38.421-0500 I CONTROL  [initandlisten] build info: Linux linux-p7yj.site 3.17.4-2.g2d23787-desktop #1 SMP PREEMPT Tue Nov 25 20:17:30 UTC 2014 (2d23787) x86_64 BOOST_LIB_VERSION=1_49
      2014-12-24T11:51:38.421-0500 I CONTROL  [initandlisten] allocator: tcmalloc
      2014-12-24T11:51:38.421-0500 I CONTROL  [initandlisten] options: { storage: { dbPath: "db1", directoryPerDB: true, repairPath: "db1/path" } }
      2014-12-24T11:51:38.436-0500 I JOURNAL  [initandlisten] journal dir=db1/journal
      2014-12-24T11:51:38.436-0500 I JOURNAL  [initandlisten] recover : no journal files present, no recovery needed
      2014-12-24T11:51:39.058-0500 I JOURNAL  [initandlisten] preallocateIsFaster=true 4.6
      2014-12-24T11:51:39.306-0500 I JOURNAL  [initandlisten] preallocateIsFaster=true 2.42
      2014-12-24T11:51:40.526-0500 I INDEX    [initandlisten] allocating new ns file db1/local/local.ns, filling with zeroes...
      2014-12-24T11:51:40.572-0500 I STORAGE  [FileAllocator] allocating new datafile db1/local/local.0, filling with zeroes...
      2014-12-24T11:51:40.573-0500 I STORAGE  [FileAllocator] creating directory db1/local/_tmp
      2014-12-24T11:51:40.578-0500 I STORAGE  [FileAllocator] done allocating datafile db1/local/local.0, size: 64MB,  took 0.002 secs
      2014-12-24T11:51:40.579-0500 I STORAGE  [initandlisten] datafileheader::init initializing db1/local/local.0 n:0
      2014-12-24T11:51:40.580-0500 I NETWORK  [initandlisten] waiting for connections on port 27017
      2014-12-24T11:51:43.468-0500 I NETWORK  [initandlisten] connection accepted from 127.0.0.1:43964 #1 (1 connection now open)
      2014-12-24T11:51:51.191-0500 I INDEX    [conn1] allocating new ns file db1/path/path.ns, filling with zeroes...
      2014-12-24T11:51:51.231-0500 I STORAGE  [FileAllocator] allocating new datafile db1/path/path.0, filling with zeroes...
      2014-12-24T11:51:51.231-0500 I STORAGE  [FileAllocator] creating directory db1/path/_tmp
      2014-12-24T11:51:51.239-0500 I STORAGE  [FileAllocator] done allocating datafile db1/path/path.0, size: 64MB,  took 0.003 secs
      2014-12-24T11:51:51.239-0500 I STORAGE  [conn1] datafileheader::init initializing db1/path/path.0 n:0
      2014-12-24T11:52:14.945-0500 I COMMAND  [conn1] repairDatabase path
      2014-12-24T11:52:14.948-0500 I STORAGE  [conn1] repairDatabase path
      2014-12-24T11:52:16.144-0500 I JOURNAL  [conn1] removeJournalFiles
      2014-12-24T11:52:16.176-0500 I INDEX    [conn1] allocating new ns file db1/path/_tmp_repairDatabase_0/path/path.ns, filling with zeroes...
      2014-12-24T11:52:16.220-0500 I STORAGE  [FileAllocator] allocating new datafile db1/path/_tmp_repairDatabase_0/path/path.0, filling with zeroes...
      2014-12-24T11:52:16.220-0500 I STORAGE  [FileAllocator] creating directory db1/path/_tmp_repairDatabase_0/path/_tmp
      2014-12-24T11:52:16.237-0500 I STORAGE  [FileAllocator] done allocating datafile db1/path/_tmp_repairDatabase_0/path/path.0, size: 64MB,  took 0.008 secs
      2014-12-24T11:52:16.238-0500 I STORAGE  [conn1] datafileheader::init initializing db1/path/_tmp_repairDatabase_0/path/path.0 n:0
      2014-12-24T11:52:16.238-0500 I INDEX    [conn1] build index on: path.col properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "path.col" }
      2014-12-24T11:52:16.238-0500 I INDEX    [conn1]          building index using bulk method
      2014-12-24T11:52:17.476-0500 I JOURNAL  [conn1] removeJournalFiles
      2014-12-24T11:52:18.077-0500 I JOURNAL  [conn1] removeJournalFiles
      2014-12-24T11:52:19.302-0500 I JOURNAL  [conn1] removeJournalFiles
      2014-12-24T11:52:19.326-0500 I INDEX    [conn1] restarting 0 index build(s)
      2014-12-24T11:52:19.326-0500 I QUERY    [conn1] command path.$cmd command: repairDatabase { repairDatabase: 1.0 } keyUpdates:0 numYields:0  reslen:37 4380ms
      > use path
      switched to db path
      > db.col.insert({doc: 0})
      WriteResult({ "nInserted" : 1 })
      > db.col.insert({doc: 1})
      WriteResult({ "nInserted" : 1 })
      > db.col.insert({doc: 2})
      WriteResult({ "nInserted" : 1 })
      > db.col.insert({doc: 3})
      WriteResult({ "nInserted" : 1 })
      > db.col.insert({doc: 4})
      WriteResult({ "nInserted" : 1 })
      > db.col.insert({doc: 5})
      WriteResult({ "nInserted" : 1 })
      > db.col.find()
      { "_id" : ObjectId("549aef270c6d2a673ccba696"), "doc" : 0 }
      { "_id" : ObjectId("549aef2a0c6d2a673ccba697"), "doc" : 1 }
      { "_id" : ObjectId("549aef2b0c6d2a673ccba698"), "doc" : 2 }
      { "_id" : ObjectId("549aef2d0c6d2a673ccba699"), "doc" : 3 }
      { "_id" : ObjectId("549aef310c6d2a673ccba69a"), "doc" : 4 }
      { "_id" : ObjectId("549aef330c6d2a673ccba69b"), "doc" : 5 }
      > db.runCommand({repairDatabase: 1})
      { "ok" : 1 }
      > db.col.find()
      
      Show
      sajack@linux-p7yj /home/sajack/mongo git SERVER-16615 () % ./mongod --dbpath db1 --repairpath db1/path --directoryperdb :) 2014-12-24T11:51:38.421-0500 I CONTROL [initandlisten] MongoDB starting : pid=6714 port=27017 dbpath=db1 64-bit host=linux-p7yj.site 2014-12-24T11:51:38.421-0500 I CONTROL [initandlisten] _DEBUG build (which is slower) 2014-12-24T11:51:38.421-0500 I CONTROL [initandlisten] 2014-12-24T11:51:38.421-0500 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. 2014-12-24T11:51:38.421-0500 I CONTROL [initandlisten] ** We suggest setting it to 'never' 2014-12-24T11:51:38.421-0500 I CONTROL [initandlisten] 2014-12-24T11:51:38.421-0500 I CONTROL [initandlisten] db version v2.8.0-rc4-pre- 2014-12-24T11:51:38.421-0500 I CONTROL [initandlisten] git version: 6d217f7a83979c441288611a06b08857eda0234c modules: subscription 2014-12-24T11:51:38.421-0500 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1j 15 Oct 2014 2014-12-24T11:51:38.421-0500 I CONTROL [initandlisten] build info: Linux linux-p7yj.site 3.17.4-2.g2d23787-desktop #1 SMP PREEMPT Tue Nov 25 20:17:30 UTC 2014 (2d23787) x86_64 BOOST_LIB_VERSION=1_49 2014-12-24T11:51:38.421-0500 I CONTROL [initandlisten] allocator: tcmalloc 2014-12-24T11:51:38.421-0500 I CONTROL [initandlisten] options: { storage: { dbPath: "db1", directoryPerDB: true, repairPath: "db1/path" } } 2014-12-24T11:51:38.436-0500 I JOURNAL [initandlisten] journal dir=db1/journal 2014-12-24T11:51:38.436-0500 I JOURNAL [initandlisten] recover : no journal files present, no recovery needed 2014-12-24T11:51:39.058-0500 I JOURNAL [initandlisten] preallocateIsFaster=true 4.6 2014-12-24T11:51:39.306-0500 I JOURNAL [initandlisten] preallocateIsFaster=true 2.42 2014-12-24T11:51:40.526-0500 I INDEX [initandlisten] allocating new ns file db1/local/local.ns, filling with zeroes... 2014-12-24T11:51:40.572-0500 I STORAGE [FileAllocator] allocating new datafile db1/local/local.0, filling with zeroes... 2014-12-24T11:51:40.573-0500 I STORAGE [FileAllocator] creating directory db1/local/_tmp 2014-12-24T11:51:40.578-0500 I STORAGE [FileAllocator] done allocating datafile db1/local/local.0, size: 64MB, took 0.002 secs 2014-12-24T11:51:40.579-0500 I STORAGE [initandlisten] datafileheader::init initializing db1/local/local.0 n:0 2014-12-24T11:51:40.580-0500 I NETWORK [initandlisten] waiting for connections on port 27017 2014-12-24T11:51:43.468-0500 I NETWORK [initandlisten] connection accepted from 127.0.0.1:43964 #1 (1 connection now open) 2014-12-24T11:51:51.191-0500 I INDEX [conn1] allocating new ns file db1/path/path.ns, filling with zeroes... 2014-12-24T11:51:51.231-0500 I STORAGE [FileAllocator] allocating new datafile db1/path/path.0, filling with zeroes... 2014-12-24T11:51:51.231-0500 I STORAGE [FileAllocator] creating directory db1/path/_tmp 2014-12-24T11:51:51.239-0500 I STORAGE [FileAllocator] done allocating datafile db1/path/path.0, size: 64MB, took 0.003 secs 2014-12-24T11:51:51.239-0500 I STORAGE [conn1] datafileheader::init initializing db1/path/path.0 n:0 2014-12-24T11:52:14.945-0500 I COMMAND [conn1] repairDatabase path 2014-12-24T11:52:14.948-0500 I STORAGE [conn1] repairDatabase path 2014-12-24T11:52:16.144-0500 I JOURNAL [conn1] removeJournalFiles 2014-12-24T11:52:16.176-0500 I INDEX [conn1] allocating new ns file db1/path/_tmp_repairDatabase_0/path/path.ns, filling with zeroes... 2014-12-24T11:52:16.220-0500 I STORAGE [FileAllocator] allocating new datafile db1/path/_tmp_repairDatabase_0/path/path.0, filling with zeroes... 2014-12-24T11:52:16.220-0500 I STORAGE [FileAllocator] creating directory db1/path/_tmp_repairDatabase_0/path/_tmp 2014-12-24T11:52:16.237-0500 I STORAGE [FileAllocator] done allocating datafile db1/path/_tmp_repairDatabase_0/path/path.0, size: 64MB, took 0.008 secs 2014-12-24T11:52:16.238-0500 I STORAGE [conn1] datafileheader::init initializing db1/path/_tmp_repairDatabase_0/path/path.0 n:0 2014-12-24T11:52:16.238-0500 I INDEX [conn1] build index on: path.col properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "path.col" } 2014-12-24T11:52:16.238-0500 I INDEX [conn1] building index using bulk method 2014-12-24T11:52:17.476-0500 I JOURNAL [conn1] removeJournalFiles 2014-12-24T11:52:18.077-0500 I JOURNAL [conn1] removeJournalFiles 2014-12-24T11:52:19.302-0500 I JOURNAL [conn1] removeJournalFiles 2014-12-24T11:52:19.326-0500 I INDEX [conn1] restarting 0 index build(s) 2014-12-24T11:52:19.326-0500 I QUERY [conn1] command path.$cmd command: repairDatabase { repairDatabase: 1.0 } keyUpdates:0 numYields:0 reslen:37 4380ms > use path switched to db path > db.col.insert({doc: 0}) WriteResult({ "nInserted" : 1 }) > db.col.insert({doc: 1}) WriteResult({ "nInserted" : 1 }) > db.col.insert({doc: 2}) WriteResult({ "nInserted" : 1 }) > db.col.insert({doc: 3}) WriteResult({ "nInserted" : 1 }) > db.col.insert({doc: 4}) WriteResult({ "nInserted" : 1 }) > db.col.insert({doc: 5}) WriteResult({ "nInserted" : 1 }) > db.col.find() { "_id" : ObjectId("549aef270c6d2a673ccba696"), "doc" : 0 } { "_id" : ObjectId("549aef2a0c6d2a673ccba697"), "doc" : 1 } { "_id" : ObjectId("549aef2b0c6d2a673ccba698"), "doc" : 2 } { "_id" : ObjectId("549aef2d0c6d2a673ccba699"), "doc" : 3 } { "_id" : ObjectId("549aef310c6d2a673ccba69a"), "doc" : 4 } { "_id" : ObjectId("549aef330c6d2a673ccba69b"), "doc" : 5 } > db.runCommand({repairDatabase: 1}) { "ok" : 1 } > db.col.find()

      It appears that mongodb uses the repairpath for scratch work during database repair, and cleans up the contents afterward. However, database files can be placed inside of the repairpath by using directoryperdb and then creating a database with the same name as the repairpath directory in the dbpath. This may cause the database to be removed after a repair operation is performed. This behavior was not seen with WiredTiger.

            Assignee:
            backlog-server-execution [DO NOT USE] Backlog - Storage Execution Team
            Reporter:
            spencer.jackson@mongodb.com Spencer Jackson
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: