Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-10447

dropDatabase() doesn't know about symlinks

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Fully Compatible
    • ALL
    • Hide

      Here is an example of the current behavior in action, using MongoDB 2.4.5:

      [user@mongo ~]$ ls -lh /var/mongo/data/
      drwxr-xr-x 2 mongo mongo 4.0K May 9 14:17 mydb01 -> /u0/mydb01
      lrwxrwxrwx 1 mongo mongo 34 May 9 14:17 mydb02 -> /u1/mydb02
      lrwxrwxrwx 1 mongo mongo 32 May 9 14:17 mydb03 -> /u2/mydb03
      drwxr-xr-x 2 mongo mongo 4.0K Aug 6 16:56 journal
      -rwxr-xr-x 1 mongo mongo 6 Jul 18 13:27 mongod.lock
      drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp

      [user@mongo ~]$ ls -lh /u0/mydb01/
      rw------ 1 mongo mongo 64M Aug 6 17:42 mydb01.0
      rw------ 1 mongo mongo 128M Aug 6 14:05 mydb01.1
      rw------ 1 mongo mongo 2.0G Aug 2 10:27 mydb01.10
      rw------ 1 mongo mongo 2.0G Aug 6 02:10 mydb01.100
      ...

      > use mydb01
      switched to db mydb01
      > db.dropDatabase()

      { "dropped" : "mydb01", "ok" : 1 }

      > db.mydb01.insert(

      {"a":"b"}

      )

      [user@mongo ~]$ ls -lh /var/mongo/data/
      lrwxrwxrwx 1 mongo mongo 34 May 9 14:17 mydb02 -> /u1/mydb02
      lrwxrwxrwx 1 mongo mongo 32 May 9 14:17 mydb03 -> /u2/mydb03
      drwxr-xr-x 2 mongo mongo 4.0K Aug 6 16:56 journal
      -rwxr-xr-x 1 mongo mongo 6 Jul 18 13:27 mongod.lock
      drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp

      [user@mongo ~]$ ls -lh /u0/mydb01/
      rw------ 1 mongo mongo 64M Aug 6 17:42 mydb01.0
      rw------ 1 mongo mongo 128M Aug 6 14:05 mydb01.1
      rw------ 1 mongo mongo 2.0G Aug 2 10:27 mydb01.10
      rw------ 1 mongo mongo 2.0G Aug 6 02:10 mydb01.100
      ...

      ----------

      Here is an example of this behavior in action, using MongoDB 2.4.5, patched to preserve
      symlinks:

      [user@mongo ~]$ ls -lh /var/mongo/data/
      drwxr-xr-x 2 mongo mongo 4.0K May 9 14:17 mydb01 -> /u0/mydb01
      lrwxrwxrwx 1 mongo mongo 34 May 9 14:17 mydb02 -> /u1/mydb02
      lrwxrwxrwx 1 mongo mongo 32 May 9 14:17 mydb03 -> /u2/mydb03
      drwxr-xr-x 2 mongo mongo 4.0K Aug 6 16:56 journal
      -rwxr-xr-x 1 mongo mongo 6 Jul 18 13:27 mongod.lock
      drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp

      [user@mongo ~]$ ls -lh /u0/mydb01/
      rw------ 1 mongo mongo 64M Aug 6 17:42 mydb01.0
      rw------ 1 mongo mongo 128M Aug 6 14:05 mydb01.1
      rw------ 1 mongo mongo 2.0G Aug 2 10:27 mydb01.10
      rw------ 1 mongo mongo 2.0G Aug 6 02:10 mydb01.100
      ...

      > use mydb01
      switched to db mydb01
      > db.dropDatabase()

      { "dropped" : "mydb01", "ok" : 1 }

      > db.mydb01.insert(

      {"a":"b"}

      )

      [user@mongo ~]$ ls -lh /var/mongo/data/
      drwxr-xr-x 2 mongo mongo 4.0K May 9 14:17 mydb01 -> /u0/mydb01
      lrwxrwxrwx 1 mongo mongo 34 May 9 14:17 mydb02 -> /u1/mydb02
      lrwxrwxrwx 1 mongo mongo 32 May 9 14:17 mydb03 -> /u2/mydb03
      drwxr-xr-x 2 mongo mongo 4.0K Aug 6 16:56 journal
      -rwxr-xr-x 1 mongo mongo 6 Jul 18 13:27 mongod.lock
      drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp

      [user@mongo ~]$ ls -lh /u0/mydb01/
      rw------ 1 mongo mongo 64M Aug 6 18:42 mydb01.0
      rw------ 1 mongo mongo 128M Aug 6 18:42 mydb01.1
      drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp

      Show
      Here is an example of the current behavior in action, using MongoDB 2.4.5: [user@mongo ~] $ ls -lh /var/mongo/data/ drwxr-xr-x 2 mongo mongo 4.0K May 9 14:17 mydb01 -> /u0/mydb01 lrwxrwxrwx 1 mongo mongo 34 May 9 14:17 mydb02 -> /u1/mydb02 lrwxrwxrwx 1 mongo mongo 32 May 9 14:17 mydb03 -> /u2/mydb03 drwxr-xr-x 2 mongo mongo 4.0K Aug 6 16:56 journal -rwxr-xr-x 1 mongo mongo 6 Jul 18 13:27 mongod.lock drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp [user@mongo ~] $ ls -lh /u0/mydb01/ rw ------ 1 mongo mongo 64M Aug 6 17:42 mydb01.0 rw ------ 1 mongo mongo 128M Aug 6 14:05 mydb01.1 rw ------ 1 mongo mongo 2.0G Aug 2 10:27 mydb01.10 rw ------ 1 mongo mongo 2.0G Aug 6 02:10 mydb01.100 ... > use mydb01 switched to db mydb01 > db.dropDatabase() { "dropped" : "mydb01", "ok" : 1 } > db.mydb01.insert( {"a":"b"} ) [user@mongo ~] $ ls -lh /var/mongo/data/ lrwxrwxrwx 1 mongo mongo 34 May 9 14:17 mydb02 -> /u1/mydb02 lrwxrwxrwx 1 mongo mongo 32 May 9 14:17 mydb03 -> /u2/mydb03 drwxr-xr-x 2 mongo mongo 4.0K Aug 6 16:56 journal -rwxr-xr-x 1 mongo mongo 6 Jul 18 13:27 mongod.lock drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp [user@mongo ~] $ ls -lh /u0/mydb01/ rw ------ 1 mongo mongo 64M Aug 6 17:42 mydb01.0 rw ------ 1 mongo mongo 128M Aug 6 14:05 mydb01.1 rw ------ 1 mongo mongo 2.0G Aug 2 10:27 mydb01.10 rw ------ 1 mongo mongo 2.0G Aug 6 02:10 mydb01.100 ... ---------- Here is an example of this behavior in action, using MongoDB 2.4.5, patched to preserve symlinks: [user@mongo ~] $ ls -lh /var/mongo/data/ drwxr-xr-x 2 mongo mongo 4.0K May 9 14:17 mydb01 -> /u0/mydb01 lrwxrwxrwx 1 mongo mongo 34 May 9 14:17 mydb02 -> /u1/mydb02 lrwxrwxrwx 1 mongo mongo 32 May 9 14:17 mydb03 -> /u2/mydb03 drwxr-xr-x 2 mongo mongo 4.0K Aug 6 16:56 journal -rwxr-xr-x 1 mongo mongo 6 Jul 18 13:27 mongod.lock drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp [user@mongo ~] $ ls -lh /u0/mydb01/ rw ------ 1 mongo mongo 64M Aug 6 17:42 mydb01.0 rw ------ 1 mongo mongo 128M Aug 6 14:05 mydb01.1 rw ------ 1 mongo mongo 2.0G Aug 2 10:27 mydb01.10 rw ------ 1 mongo mongo 2.0G Aug 6 02:10 mydb01.100 ... > use mydb01 switched to db mydb01 > db.dropDatabase() { "dropped" : "mydb01", "ok" : 1 } > db.mydb01.insert( {"a":"b"} ) [user@mongo ~] $ ls -lh /var/mongo/data/ drwxr-xr-x 2 mongo mongo 4.0K May 9 14:17 mydb01 -> /u0/mydb01 lrwxrwxrwx 1 mongo mongo 34 May 9 14:17 mydb02 -> /u1/mydb02 lrwxrwxrwx 1 mongo mongo 32 May 9 14:17 mydb03 -> /u2/mydb03 drwxr-xr-x 2 mongo mongo 4.0K Aug 6 16:56 journal -rwxr-xr-x 1 mongo mongo 6 Jul 18 13:27 mongod.lock drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp [user@mongo ~] $ ls -lh /u0/mydb01/ rw ------ 1 mongo mongo 64M Aug 6 18:42 mydb01.0 rw ------ 1 mongo mongo 128M Aug 6 18:42 mydb01.1 drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp

      First of all, thanks for developing MongoDB, and releasing it as open source software!

      We have been using MongoDB with the --directoryperdb option, and it's been running great. As the data in the system has expanded, we added some additional hard drives to the server, to reduce spindle contention and increase the amount o
      f disk space available. To accomplish this, we used a simple method of creating symlinks from the Mongo data directory out to per-database directories on separate drives. Others online have done similar things before us (Google: mongodb directoryperdb symlink).

      Recently, we discovered that if you drop such a database from the Mongo shell, Mongo will simply remove the symlink, without dereferencing it. The end result is that the symlink goes away, and the data on the additional drive becomes orphaned. Worse, if you later recreate or otherwise try to access the same database again, Mongo has no memory of the symlink or the other drive, and creates the database anew.

      The end result is that under this configuration, if you drop and then re-add such a database, the new data will be written to the original drive.

      I have attached a patch to MongoDB 2.4.5 that changes this behavior. In normal circumstances, it should work just as it did before. However, in the situation described above, MongoDB will instead follow the symlink, and remove the contents of the target directory instead.

      The theory is that since the symlink was placed there by a system administrator, it will be left alone. This makes creating and deleting a database a symmetrical operation from the Mongo shell. Creating and deleting the symlink and provisioning additional hard drives is also a symmetrical operation for the system administrator.

        1. mongodb-src-r2.4.5-preserve-directoryperdb-symlinks.patch
          3 kB

            Assignee:
            Unassigned Unassigned
            Reporter:
            nrosenqu Nathan Rosenquist
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: