[SERVER-1217] Invalid cross-device link when repairDatabase() is called with --repairpath using a path on another file system Created: 11/Jun/10  Updated: 12/Jul/16  Resolved: 07/Jul/10

Status: Closed
Project: Core Server
Component/s: Tools
Affects Version/s: 1.4.3
Fix Version/s: 1.5.5

Type: Bug Priority: Critical - P2
Reporter: Erwan Arzur Assignee: Aaron Staple
Resolution: Done Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:34:28 EST 2008 x86_64 x86_64 x86_64 GNU/Linux
/mongo/master on 8x70GB EBS volumes
Database size ~400GB


Operating System: Linux
Participants:

 Description   
  1. /opt/mongo/bin/mongod --dbpath=/mongo/master --repairpath=/mnt/mongo --repair
    Fri Jun 11 12:52:58 Mongo DB : starting : pid = 23801 port = 27017 dbpath = /mongo/master master = 0 slave = 0 64-bit
    Fri Jun 11 12:52:58 ****
    Fri Jun 11 12:52:58 ****
    Fri Jun 11 12:52:58 need to upgrade database local with pdfile version 4.5, new version: 4.5
    Fri Jun 11 12:52:58 starting upgrade
    Fri Jun 11 12:52:58 local repairDatabase local
    Fri Jun 11 12:52:58 allocating new datafile /mnt/mongo/$tmp_repairDatabase_0/local.ns, filling with zeroes...
    Fri Jun 11 12:52:58 done allocating datafile /mnt/mongo/$tmp_repairDatabase_0/local.ns, size: 16MB, took 0.011 secs
    Fri Jun 11 12:52:58 allocating new datafile /mnt/mongo/$tmp_repairDatabase_0/local.0, filling with zeroes...
    Fri Jun 11 12:52:58 done allocating datafile /mnt/mongo/$tmp_repairDatabase_0/local.0, size: 64MB, took 0.044 secs
    Fri Jun 11 12:52:58 allocating new datafile /mnt/mongo/$tmp_repairDatabase_0/local.1, filling with zeroes...
    Fri Jun 11 12:52:58 building new index on { _id: 1 }

    for local.sources
    Fri Jun 11 12:52:58 Buildindex local.sources idxNo:0

    Unknown macro: { name}

    Fri Jun 11 12:52:58 done for 0 records 0.028secs
    Fri Jun 11 12:52:58 building new index on

    { _id: 1 }

    for local.pair.sync
    Fri Jun 11 12:52:58 Buildindex local.pair.sync idxNo:0

    Unknown macro: { name}

    Fri Jun 11 12:52:58 done allocating datafile /mnt/mongo/$tmp_repairDatabase_0/local.1, size: 128MB, took 0.102 secs
    Fri Jun 11 12:52:58 done for 0 records 0.012secs
    Fri Jun 11 12:52:58 caught boost exception: boost::filesystem::rename: Invalid cross-device link: "/mnt/mongo/$tmp_repairDatabase_0/local.ns", "/mongo/master/local.ns"
    Fri Jun 11 12:52:58 Assertion failure false db/pdfile.cpp 1769
    0x4f6986 0x4ff094 0x602c94 0x602de8 0x603558 0x6acced 0x6ae401 0x6aeeed 0x6af715 0x6b51d2 0x35cd01e32a 0x4a17d9
    /opt/mongo/bin/mongod(_ZN5mongo12sayDbContextEPKc+0xe6) [0x4f6986]
    /opt/mongo/bin/mongod(_ZN5mongo8assertedEPKcS1_j+0x154) [0x4ff094]
    /opt/mongo/bin/mongod(_ZN5mongo19_applyOpToDataFilesEPKcRNS_6FileOpEbRKSs+0xaf4) [0x602c94]
    /opt/mongo/bin/mongod(ZN5mongo21_replaceWithRecoveredEPKcS1+0x98) [0x602de8]
    /opt/mongo/bin/mongod(_ZN5mongo14repairDatabaseEPKcRSsbb+0x648) [0x603558]
    /opt/mongo/bin/mongod(_ZN5mongo11doDBUpgradeERKSsSsPNS_9MDFHeaderE+0x4d) [0x6acced]
    /opt/mongo/bin/mongod(_ZN5mongo15repairDatabasesEv+0x2b1) [0x6ae401]
    /opt/mongo/bin/mongod(_ZN5mongo14_initAndListenEiPKc+0x44d) [0x6aeeed]
    /opt/mongo/bin/mongod(_ZN5mongo13initAndListenEiPKc+0x25) [0x6af715]
    /opt/mongo/bin/mongod(main+0x5172) [0x6b51d2]
    /lib64/libc.so.6(__libc_start_main+0xfa) [0x35cd01e32a]
    /opt/mongo/bin/mongod(_ZNSt15basic_streambufIcSt11char_traitsIcEE6xsputnEPKcl+0x69) [0x4a17d9]
    Fri Jun 11 12:52:58 exception in initAndListen std::exception: assertion db/pdfile.cpp:1769, terminating
    Fri Jun 11 12:52:58 dbexit:
    Fri Jun 11 12:52:58 shutdown: going to close listening sockets...
    Fri Jun 11 12:52:58 shutdown: going to flush oplog...
    Fri Jun 11 12:52:58 shutdown: going to close sockets...
    Fri Jun 11 12:52:58 shutdown: waiting for fs preallocator...
    Fri Jun 11 12:52:58 shutdown: closing all files...
    Fri Jun 11 12:52:58 closeAllFiles() finished
    Fri Jun 11 12:52:58 shutdown: removing fs lock...
    Fri Jun 11 12:52:58 dbexit: really exiting now



 Comments   
Comment by auto [ 06/Jul/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 use less space
http://github.com/mongodb/mongo/commit/d62e21a084b61d1a73360eefc069bdd7857c0e88

Comment by auto [ 06/Jul/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 old boost?
http://github.com/mongodb/mongo/commit/e3a992d4a37c3f05b407faaaa6413a9508521c89

Comment by auto [ 06/Jul/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 work old boost
http://github.com/mongodb/mongo/commit/3040636830489f5788604957f7fca0499630b94f

Comment by auto [ 06/Jul/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 try smaller nssize with bigger partition
http://github.com/mongodb/mongo/commit/07cc7b450b96cb9b0118dd7d10788ac9512cffbb

Comment by Michael Dirolf [ 06/Jul/10 ]

Can you try again now and let me know if that's working? That directory should be a 32MB loopback device now.

Comment by Michael Dirolf [ 06/Jul/10 ]

going to just setup a loopback device instead - will let you know when it's up

Comment by Aaron Staple [ 30/Jun/10 ]

sorry to bother you again Mike - the ramdisk needs to be at least 32mb

Comment by auto [ 30/Jun/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 disable test
http://github.com/mongodb/mongo/commit/66d07d51598b0e7d8fb9da99305f223a4df46e85

Comment by auto [ 30/Jun/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 small files
http://github.com/mongodb/mongo/commit/6888b8e403a9f890b4a869e0a5fa8a878a546e10

Comment by auto [ 30/Jun/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 path
http://github.com/mongodb/mongo/commit/084e0b81737cd064948a87ba9f568b574e661305

Comment by auto [ 29/Jun/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 subdir
http://github.com/mongodb/mongo/commit/6af4ec3ef32bce3851e6e3d09609945d72c22483

Comment by Michael Dirolf [ 29/Jun/10 ]

It looks like the test is trying to remove the entire directory? This is just a mounted ramdisk, so that's always going to give an error. Any way to just remove the files in the directory? or use a subdir as the repair path?

Comment by auto [ 28/Jun/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 disable test
http://github.com/mongodb/mongo/commit/2abad27e872f373443c3670775b8c2c49e655c70

Comment by Aaron Staple [ 28/Jun/10 ]

I wrote a test and it worked on my machine. Not sure why the device or resource was busy on the buildbot. Mike can you take a look?

<http://buildbot.mongodb.org/builders/Linux%2064-bit%20Legacy/builds/1299/steps/test/logs/stdio>

Comment by auto [ 28/Jun/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 add test
http://github.com/mongodb/mongo/commit/344a92341e7537ca86cd009c92bcbdb1f5e9d344

Comment by Aaron Staple [ 28/Jun/10 ]

sorry, above commit is actually for a different jira

Comment by auto [ 28/Jun/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 bestGuessCursor() should always honor sort spec
http://github.com/mongodb/mongo/commit/6dd05ca594e4a93fdc827da89e39986e188fed5f

Comment by Michael Dirolf [ 25/Jun/10 ]

I've setup a small (16M) ramdisk partition on ofc-n1:
/data/db/repairpartitiontest/

Do you already have a test that I should add to buildbot for this or did you want me to try to put something together?

Comment by auto [ 21/Jun/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 old boost?
http://github.com/mongodb/mongo/commit/b543b4a4a8ea7e77a235f15c55afe6e46d15f2af

Comment by Aaron Staple [ 21/Jun/10 ]

Mike, let me know if I have assigned to you incorrectly. I know you do some work on the buildslaves.

Comment by Aaron Staple [ 21/Jun/10 ]

would be nice to have a buildbot test for this. Can we set up a separate partition on one of the buildslaves to test?

Comment by auto [ 21/Jun/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-1217 boost rename doesn't work across partitions, so copy and then delete
http://github.com/mongodb/mongo/commit/2353944ea0a68fd27d33fed05f211546fb51b515

Generated at Thu Feb 08 02:56:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.