[SERVER-10447] dropDatabase() doesn't know about symlinks Created: 07/Aug/13  Updated: 09/Jul/16  Resolved: 02/Mar/14

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Nathan Rosenquist Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File mongodb-src-r2.4.5-preserve-directoryperdb-symlinks.patch    
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

Here is an example of the current behavior in action, using MongoDB 2.4.5:

[user@mongo ~]$ ls -lh /var/mongo/data/
drwxr-xr-x 2 mongo mongo 4.0K May 9 14:17 mydb01 -> /u0/mydb01
lrwxrwxrwx 1 mongo mongo 34 May 9 14:17 mydb02 -> /u1/mydb02
lrwxrwxrwx 1 mongo mongo 32 May 9 14:17 mydb03 -> /u2/mydb03
drwxr-xr-x 2 mongo mongo 4.0K Aug 6 16:56 journal
-rwxr-xr-x 1 mongo mongo 6 Jul 18 13:27 mongod.lock
drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp

[user@mongo ~]$ ls -lh /u0/mydb01/
rw------ 1 mongo mongo 64M Aug 6 17:42 mydb01.0
rw------ 1 mongo mongo 128M Aug 6 14:05 mydb01.1
rw------ 1 mongo mongo 2.0G Aug 2 10:27 mydb01.10
rw------ 1 mongo mongo 2.0G Aug 6 02:10 mydb01.100
...

> use mydb01
switched to db mydb01
> db.dropDatabase()

{ "dropped" : "mydb01", "ok" : 1 }

> db.mydb01.insert(

{"a":"b"}

)

[user@mongo ~]$ ls -lh /var/mongo/data/
lrwxrwxrwx 1 mongo mongo 34 May 9 14:17 mydb02 -> /u1/mydb02
lrwxrwxrwx 1 mongo mongo 32 May 9 14:17 mydb03 -> /u2/mydb03
drwxr-xr-x 2 mongo mongo 4.0K Aug 6 16:56 journal
-rwxr-xr-x 1 mongo mongo 6 Jul 18 13:27 mongod.lock
drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp

[user@mongo ~]$ ls -lh /u0/mydb01/
rw------ 1 mongo mongo 64M Aug 6 17:42 mydb01.0
rw------ 1 mongo mongo 128M Aug 6 14:05 mydb01.1
rw------ 1 mongo mongo 2.0G Aug 2 10:27 mydb01.10
rw------ 1 mongo mongo 2.0G Aug 6 02:10 mydb01.100
...

----------

Here is an example of this behavior in action, using MongoDB 2.4.5, patched to preserve
symlinks:

[user@mongo ~]$ ls -lh /var/mongo/data/
drwxr-xr-x 2 mongo mongo 4.0K May 9 14:17 mydb01 -> /u0/mydb01
lrwxrwxrwx 1 mongo mongo 34 May 9 14:17 mydb02 -> /u1/mydb02
lrwxrwxrwx 1 mongo mongo 32 May 9 14:17 mydb03 -> /u2/mydb03
drwxr-xr-x 2 mongo mongo 4.0K Aug 6 16:56 journal
-rwxr-xr-x 1 mongo mongo 6 Jul 18 13:27 mongod.lock
drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp

[user@mongo ~]$ ls -lh /u0/mydb01/
rw------ 1 mongo mongo 64M Aug 6 17:42 mydb01.0
rw------ 1 mongo mongo 128M Aug 6 14:05 mydb01.1
rw------ 1 mongo mongo 2.0G Aug 2 10:27 mydb01.10
rw------ 1 mongo mongo 2.0G Aug 6 02:10 mydb01.100
...

> use mydb01
switched to db mydb01
> db.dropDatabase()

{ "dropped" : "mydb01", "ok" : 1 }

> db.mydb01.insert(

{"a":"b"}

)

[user@mongo ~]$ ls -lh /var/mongo/data/
drwxr-xr-x 2 mongo mongo 4.0K May 9 14:17 mydb01 -> /u0/mydb01
lrwxrwxrwx 1 mongo mongo 34 May 9 14:17 mydb02 -> /u1/mydb02
lrwxrwxrwx 1 mongo mongo 32 May 9 14:17 mydb03 -> /u2/mydb03
drwxr-xr-x 2 mongo mongo 4.0K Aug 6 16:56 journal
-rwxr-xr-x 1 mongo mongo 6 Jul 18 13:27 mongod.lock
drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp

[user@mongo ~]$ ls -lh /u0/mydb01/
rw------ 1 mongo mongo 64M Aug 6 18:42 mydb01.0
rw------ 1 mongo mongo 128M Aug 6 18:42 mydb01.1
drwxr-xr-x 3 mongo mongo 4.0K Aug 6 16:56 _tmp

Participants:

 Description   

First of all, thanks for developing MongoDB, and releasing it as open source software!

We have been using MongoDB with the --directoryperdb option, and it's been running great. As the data in the system has expanded, we added some additional hard drives to the server, to reduce spindle contention and increase the amount o
f disk space available. To accomplish this, we used a simple method of creating symlinks from the Mongo data directory out to per-database directories on separate drives. Others online have done similar things before us (Google: mongodb directoryperdb symlink).

Recently, we discovered that if you drop such a database from the Mongo shell, Mongo will simply remove the symlink, without dereferencing it. The end result is that the symlink goes away, and the data on the additional drive becomes orphaned. Worse, if you later recreate or otherwise try to access the same database again, Mongo has no memory of the symlink or the other drive, and creates the database anew.

The end result is that under this configuration, if you drop and then re-add such a database, the new data will be written to the original drive.

I have attached a patch to MongoDB 2.4.5 that changes this behavior. In normal circumstances, it should work just as it did before. However, in the situation described above, MongoDB will instead follow the symlink, and remove the contents of the target directory instead.

The theory is that since the symlink was placed there by a system administrator, it will be left alone. This makes creating and deleting a database a symmetrical operation from the Mongo shell. Creating and deleting the symlink and provisioning additional hard drives is also a symmetrical operation for the system administrator.



 Comments   
Comment by Daniel Pasette (Inactive) [ 02/Mar/14 ]

I'm marking this a duplicate of SERVER-1379. Thanks for your patch Nathan – I'll make sure it's referenced in the open ticket.

Comment by Nathan Rosenquist [ 07/Aug/13 ]

Thanks for the detailed reply Stephen.

I'll have to revisit this later when I'm not on a deadline.

Thanks,
-Nathan

Comment by Stennie Steneker (Inactive) [ 07/Aug/13 ]

Hi Nathan,

Thank you for raising this issue and including a code contribution. Have you read our guide to Contributing to the MongoDB project?

There are a few extra steps that will help this request be ready for review by our kernel team:

  • add a reference to this Jira ticket in the commit comment
  • add test cases
  • adhere to the Kernel Code Style; in particular, there are some long lines added (should be limited to 100 columns)
  • sign the contributor agreement if you haven't already

Rather than providing a patch file, contributions are normally done via a pull request on Github:

  • fork the mongodb repo on Github
  • create a new feature branch for your changes
  • squash your commits once you have finished making changes
  • submit a pull request (as above, be sure to reference this SERVER issue in the commit message)

Regards,
Stephen

Comment by Nathan Rosenquist [ 07/Aug/13 ]

FYI, It seems this issue is somewhat related to SERVER-1379 as well.

Thanks again!

Generated at Thu Feb 08 03:23:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.