ISSUE SUMMARY
When moving a collection across databases, the renameCollection command does not issue a journal commit until after the data copy has finished. As a result, moving a collection of size ~0.5GB or larger across databases will trigger a fatal assertion when the journal commit buffer exceeds its maximum size.
USER IMPACT
Users running with journaling enabled can't move large collections across databases.
WORKAROUNDS
If the destination database is empty, a possible workaround is a follows:
- use the copydb command to copy all collections to the target database
- use the drop command to remove unneeded collections
- use the renameCollection command to give the collection the desired name in the target database.
If the destination database is not empty there are no workarounds.
AFFECTED VERSIONS
MongoDB 2.6.0 and 2.6.1 are affected by this issue.
FIX VERSION
The fix is included in the 2.6.2 production release.
RESOLUTION DETAILS
The renameCollection command issues a journal commit after each data insertion in the target collection.
Original description
When moving a collection across databases, the renameCollection command does not issue a journal commit until after the data copy has finished. As a result, moving a collection of size ~0.5GB or larger across databases will trigger a fatal assertion when the journal commit buffer exceeds its maximum size.
Regression since 2.4 (introduced in 0abf27ae); confirmed against 2.6.1 and 2.7.1.
Reproduce with the following:
var s = ''; for (var i=0; i<1024; i++) { s += 'x'; } for (var i=0; i<1000*1000; i++) { db.foo.insert({a: s}); } db.adminCommand({renameCollection: "test.foo", to: "test2.foo"})
The above generates the following output in the mongod log:
2014-05-27T12:13:25.641-0400 [conn2] test2.foo Assertion failure a <= 512*1024*1024 src/mongo/util/alignedbuilder.cpp 104 2014-05-27T12:13:25.656-0400 [conn2] test2.foo 0x1006ac7ab 0x1006642c2 0x100654d52 0x1006533f9 0x1001e1dd0 0x1001e2081 0x1001d84f0 0x1001d65a1 0x1001d8159 0x1001d6628 0x1000fb54b 0x1000fc26d 0x1000f6bd5 0x10016eea1 0x1001b7605 0x1001b8375 0x1001b907c 0x1003cfc7f 0x1002a1050 0x100006854 0 mongod261 0x00000001006ac7ab _ZN5mongo15printStackTraceERSo + 43 1 mongod261 0x00000001006642c2 _ZN5mongo10logContextEPKc + 114 2 mongod261 0x0000000100654d52 _ZN5mongo12verifyFailedEPKcS1_j + 274 3 mongod261 0x00000001006533f9 _ZN5mongo14AlignedBuilder14growReallocateEj + 191 4 mongod261 0x00000001001e1dd0 _ZN5mongo3durL21prepBasicWrite_inlockERNS_14AlignedBuilderEPKNS0_11WriteIntentERNS_12RelativePathE + 880 5 mongod261 0x00000001001e2081 _ZN5mongo3dur13PREPLOGBUFFERERNS0_11JSectHeaderERNS_14AlignedBuilderE + 401 6 mongod261 0x00000001001d84f0 _ZN5mongo3durL11groupCommitEPNS_4Lock11GlobalWriteE + 272 7 mongod261 0x00000001001d65a1 _ZN5mongo3dur11DurableImpl9commitNowEv + 21 8 mongod261 0x00000001001d8159 _ZN5mongo3dur11DurableImpl16_aCommitIsNeededEv + 113 9 mongod261 0x00000001001d6628 _ZN5mongo3dur11DurableImpl14commitIfNeededEb + 76 10 mongod261 0x00000001000fb54b _ZN5mongo18addExistingToIndexEPNS_10CollectionEPKNS_15IndexDescriptorEPNS_17IndexAccessMethodEb + 1275 11 mongod261 0x00000001000fc26d _ZN5mongo12buildAnIndexEPNS_10CollectionEPNS_17IndexCatalogEntryEb + 2253 12 mongod261 0x00000001000f6bd5 _ZN5mongo12IndexCatalog11createIndexENS_7BSONObjEbNS0_16ShutdownBehaviorE + 1231 13 mongod261 0x000000010016eea1 _ZN5mongo19CmdRenameCollection3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb + 7681 14 mongod261 0x00000001001b7605 _ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb + 37 15 mongod261 0x00000001001b8375 _ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 2873 16 mongod261 0x00000001001b907c _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 1388 17 mongod261 0x00000001003cfc7f _ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 1615 18 mongod261 0x00000001002a1050 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 1968 19 mongod261 0x0000000100006854 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 308 2014-05-27T12:13:25.694-0400 [conn2] dbexception in groupCommit causing immediate shutdown: 0 assertion src/mongo/util/alignedbuilder.cpp:104 2014-05-27T12:13:25.746-0400 [conn2] SEVERE: gc1 2014-05-27T12:13:25.756-0400 [conn2] SEVERE: Got signal: 6 (Abort trap: 6). Backtrace:0x1006ac7ab 0x1006ac50f 0x101cfb92a 0x1d12866 0x101d52dfa 0x100298b98 0x1001d873b 0x1001d65a1 0x1001d8159 0x1001d6628 0x1000fb54b 0x1000fc26d 0x1000f6bd5 0x10016eea1 0x1001b7605 0x1001b8375 0x1001b907c 0x1003cfc7f 0x1002a1050 0x100006854 0 mongod261 0x00000001006ac7ab _ZN5mongo15printStackTraceERSo + 43 1 mongod261 0x00000001006ac50f _ZN5mongo12_GLOBAL__N_110abruptQuitEi + 191 2 libsystem_c.dylib 0x0000000101cfb92a _sigtramp + 26 3 ??? 0x0000000001d12866 0x0 + 30484582 4 libsystem_c.dylib 0x0000000101d52dfa abort + 143 5 mongod261 0x0000000100298b98 _ZN5mongo10mongoAbortEPKc + 104 6 mongod261 0x00000001001d873b _ZN5mongo3durL11groupCommitEPNS_4Lock11GlobalWriteE + 859 7 mongod261 0x00000001001d65a1 _ZN5mongo3dur11DurableImpl9commitNowEv + 21 8 mongod261 0x00000001001d8159 _ZN5mongo3dur11DurableImpl16_aCommitIsNeededEv + 113 9 mongod261 0x00000001001d6628 _ZN5mongo3dur11DurableImpl14commitIfNeededEb + 76 10 mongod261 0x00000001000fb54b _ZN5mongo18addExistingToIndexEPNS_10CollectionEPKNS_15IndexDescriptorEPNS_17IndexAccessMethodEb + 1275 11 mongod261 0x00000001000fc26d _ZN5mongo12buildAnIndexEPNS_10CollectionEPNS_17IndexCatalogEntryEb + 2253 12 mongod261 0x00000001000f6bd5 _ZN5mongo12IndexCatalog11createIndexENS_7BSONObjEbNS0_16ShutdownBehaviorE + 1231 13 mongod261 0x000000010016eea1 _ZN5mongo19CmdRenameCollection3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb + 7681 14 mongod261 0x00000001001b7605 _ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb + 37 15 mongod261 0x00000001001b8375 _ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 2873 16 mongod261 0x00000001001b907c _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 1388 17 mongod261 0x00000001003cfc7f _ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 1615 18 mongod261 0x00000001002a1050 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 1968 19 mongod261 0x0000000100006854 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 308
Original description:
Hi,
We have a large data load that loads data from other systems in to MongoDB. At the end of the load it switched the data live by renaming the collection.
db.runCommand({renameCollection:"contacts_load.contacts_new", to:"contacts.contacts",dropTarget:true})In version 2.4.5 this has been working without a problem.
However, in version 2.6.1 this crashes the MongoD.
Please help.
Cheers
Simon
- is related to
-
SERVER-14068 Consolidate "copy collection" logic
-
- Closed
-