[SERVER-38764] External sorter should use 64-bit integers for file offsets Created: 22/Dec/18  Updated: 29/Oct/23  Resolved: 18/Jan/19

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: 4.1.6
Fix Version/s: 3.4.21, 3.6.12, 4.0.7, 4.1.8

Type: Bug Priority: Major - P3
Reporter: O Assignee: Louis Williams
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
related to SERVER-17010 Reduce file handle usage in File base... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.0, v3.6
Steps To Reproduce:
  • Create a collection
  • Insert 1.2bi documents
  • Create an index on a string or string array property of a document.
Sprint: Storage NYC 2019-01-28
Participants:

 Description   

I've a big collection, about 1.2bi documents, and I tried to create an index on this collection. I reproduced this bug with several fields.

  • I'm using 4.1.6 version on docker, running in a Linux 4.9.0-8 server.
  • This error happens even if starting the database in repair mode
  • I didn't test inserting documents with the index already on the collection. I will need about 80 hours to reproduce if you need me to test this behavior.
  • Marked the bug as blocking because other people with the same kind of bug won't be able to use their data, in a collection of this size, is not feasible to be used without an index.

Bellow the logs of the crash:

 

2018-12-22T21:25:04.993+0000 I INDEX [conn8] build index on: dashboard.delivery properties: { v: 2, unique: true, key:

{ address: 1.0 }

, name: "address_1", ns: "dashboard.delivery" }
2018-12-22T21:25:05.003+0000 I INDEX [conn8] building index using bulk method; build may temporarily use up to 500 megabytes of RAM
2018-12-22T21:25:08.003+0000 I - [conn8] Index Build: 3735700/1271300000 0%
2018-12-22T21:25:11.003+0000 I - [conn8] Index Build: 7304300/1271300000 0%
...................
2018-12-22T22:01:05.006+0000 I - [conn8] Index Build: 1263292400/1271300000 99%
2018-12-22T22:01:08.003+0000 I - [conn8] Index Build: 1266345200/1271300000 99%
2018-12-22T22:01:17.474+0000 I - [conn8] Index Build: 1268605900/1271300000 99%
2018-12-22T22:01:20.003+0000 I - [conn8] Index Build: 1270923800/1271300000 99%
2018-12-22T22:01:32.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 17998500/1271300000 1%
2018-12-22T22:01:42.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 40824000/1271300000 3%
2018-12-22T22:01:52.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 63361700/1271300000 4%
2018-12-22T22:02:02.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 84591300/1271300000 6%
2018-12-22T22:02:12.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 108159600/1271300000 8%
2018-12-22T22:02:22.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 129773100/1271300000 10%
2018-12-22T22:02:32.018+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 152750600/1271300000 12%
2018-12-22T22:02:42.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 174621400/1271300000 13%
2018-12-22T22:02:52.007+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 197153900/1271300000 15%
2018-12-22T22:03:02.008+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 219340800/1271300000 17%
2018-12-22T22:03:12.007+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 241072500/1271300000 18%
2018-12-22T22:03:22.007+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 262647400/1271300000 20%
2018-12-22T22:03:32.019+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 284618500/1271300000 22%
2018-12-22T22:03:42.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 306719600/1271300000 24%
2018-12-22T22:03:52.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 329286200/1271300000 25%
2018-12-22T22:04:02.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 351621900/1271300000 27%
2018-12-22T22:04:12.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 374338500/1271300000 29%
2018-12-22T22:04:22.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 396971400/1271300000 31%
2018-12-22T22:04:32.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 419309500/1271300000 32%
2018-12-22T22:04:42.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 439332200/1271300000 34%
2018-12-22T22:04:52.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 461051300/1271300000 36%
2018-12-22T22:05:02.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 483465300/1271300000 38%
2018-12-22T22:05:12.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 505852400/1271300000 39%
2018-12-22T22:05:22.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 528176400/1271300000 41%
2018-12-22T22:05:32.003+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 550842500/1271300000 43%
2018-12-22T22:05:42.007+0000 I - [conn8] Index: (2/3) BTree Bottom Up Progress: 562153500/1271300000 44%
2018-12-22T22:05:49.186+0000 F - [conn8] Invariant failure static_cast<unsigned int>(_file.tellg()) == _fileEndOffset src/mongo/db/sorter/sorter.cpp 293
2018-12-22T22:05:49.218+0000 F - [conn8]

***aborting after invariant() failure

2018-12-22T22:05:49.597+0000 F - [conn8] Got signal: 6 (Aborted).
0x56187d1b1ef1 0x56187d1b1109 0x56187d1b15ed 0x7f3aa87e5390 0x7f3aa843f428 0x7f3aa844102a 0x56187b7843a3 0x56187c7fea78 0x56187c7feae1 0x56187c7ff0cb 0x56187c7fc91f 0x56187c7fa718 0x56187c002128 0x56187c002424 0x56187c001308 0x56187becc034 0x56187cb38356 0x56187cb3b534 0x56187bbf8e34 0x56187bbfb1ae 0x56187bbfc94c 0x56187bbfd926 0x56187bbe9b6a 0x56187bbf5f73 0x56187bbf1767 0x56187bbf5151 0x56187c95c0f2 0x56187bbef840 0x56187bbf2b85 0x56187bbf0d9f 0x56187bbf17ed 0x56187bbf5151 0x56187c95c665 0x56187d1077d4 0x7f3aa87db6ba 0x7f3aa851141d
----- BEGIN BACKTRACE -----

{"backtrace":[

{"b":"56187AD3F000","o":"2472EF1","s":"_ZN5mongo15printStackTraceERSo"}

,{"b":"56187AD3F000","o":"2472109"},{"b":"56187AD3F000","o":"24725ED"},{"b":"7F3AA87D4000","o":"11390"},{"b":"7F3AA840A000","o":"35428","s":"gsignal"},{"b":"7F3AA840A000","o":"3702A","s":"abort"},

{"b":"56187AD3F000","o":"A453A3","s":"_ZN5mongo22invariantFailedWithMsgEPKcRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES1_j"}

,{"b":"56187AD3F000","o":"1ABFA78","s":"_ZN5mongo6sorter12FileIteratorINS_7BSONObjENS_8RecordIdEE4readEPvm"},{"b":"56187AD3F000","o":"1ABFAE1","s":"_ZN5mongo6sorter12FileIteratorINS_7BSONObjENS_8RecordIdEE18fillBufferFromDiskEv"},{"b":"56187AD3F000","o":"1AC00CB","s":"_ZN5mongo6sorter12FileIteratorINS_7BSONObjENS_8RecordIdEE4moreEv"},{"b":"56187AD3F000","o":"1ABD91F","s":"_ZN5mongo6sorter13MergeIteratorINS_7BSONObjENS_8RecordIdENS_27BtreeExternalSortComparisonEE4moreEv"},{"b":"56187AD3F000","o":"1ABB718","s":"_ZN5mongo25AbstractIndexAccessMethod10commitBulkEPNS_16OperationContextEPNS_17IndexAccessMethod11BulkBuilderEbbPSt3setINS_8RecordIdESt4lessIS7_ESaIS7_EEPSt6vectorINS_7BSONObjESaISE_EE"},{"b":"56187AD3F000","o":"12C3128","s":"_ZN5mongo19MultiIndexBlockImpl20_dumpInsertsFromBulkEPSt3setINS_8RecordIdESt4lessIS2_ESaIS2_EEPSt6vectorINS_7BSONObjESaIS9_EE"},{"b":"56187AD3F000","o":"12C3424","s":"_ZN5mongo19MultiIndexBlockImpl19dumpInsertsFromBulkEv"},{"b":"56187AD3F000","o":"12C2308","s":"_ZN5mongo19MultiIndexBlockImpl30insertAllDocumentsInCollectionEv"},{"b":"56187AD3F000","o":"118D034"},

{"b":"56187AD3F000","o":"1DF9356","s":"_ZN5mongo23ErrmsgCommandDeprecated3runEPNS_16OperationContextERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKNS_7BSONObjERNS_14BSONObjBuilderE"}

,{"b":"56187AD3F000","o":"1DFC534","s":"_ZN5mongo12BasicCommand10Invocation3runEPNS_16OperationContextEPNS_3rpc21ReplyBuilderInterfaceE"},{"b":"56187AD3F000","o":"EB9E34"},{"b":"56187AD3F000","o":"EBC1AE"},{"b":"56187AD3F000","o":"EBD94C"},{"b":"56187AD3F000","o":"EBE926","s":"_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE"},{"b":"56187AD3F000","o":"EAAB6A","s":"_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE"},{"b":"56187AD3F000","o":"EB6F73","s":"_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE"},{"b":"56187AD3F000","o":"EB2767","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"56187AD3F000","o":"EB6151"},{"b":"56187AD3F000","o":"1C1D0F2","s":"_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE"},{"b":"56187AD3F000","o":"EB0840","s":"_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE"},{"b":"56187AD3F000","o":"EB3B85","s":"_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE"},{"b":"56187AD3F000","o":"EB1D9F","s":"_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE"},{"b":"56187AD3F000","o":"EB27ED","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"56187AD3F000","o":"EB6151"},{"b":"56187AD3F000","o":"1C1D665"},{"b":"56187AD3F000","o":"23C87D4"},{"b":"7F3AA87D4000","o":"76BA"},{"b":"7F3AA840A000","o":"10741D","s":"clone"}],"processInfo":{ "mongodbVersion" : "4.1.6", "gitVersion" : "55e72b015e2aa7297c00db29e4d93451ea61a7ca", "compiledModules" : [], "uname" :

{ "sysname" : "Linux", "release" : "4.9.0-8-amd64", "version" : "#1 SMP Debian 4.9.130-2 (2018-10-27)", "machine" : "x86_64" }

, "somap" : [ { "b" : "56187AD3F000", "elfType" : 3, "buildId" : "107FA13E3E2D27E599436B56147F0382ADA8826C" }, { "b" : "7FFFFAFA0000", "elfType" : 3, "buildId" : "0B1047D7814362A9CE0444D05F6284EDA613920B" }, { "b" : "7F3AA9BE4000", "path" : "/usr/lib/x86_64-linux-gnu/libcurl.so.4", "elfType" : 3, "buildId" : "E2D1DB1857AE8F07908959381751796E89FF56BB" }, { "b" : "7F3AA99C9000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "6EF73266978476EF9F2FD2CF31E57F4597CB74F8" }, { "b" : "7F3AA9585000", "path" : "/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "250E875F74377DFC74DE48BF80CCB237BB4EFF1D" }, { "b" : "7F3AA931C000", "path" : "/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "513282AC7EB386E2C0133FD9E1B6B8A0F38B047D" }, { "b" : "7F3AA9118000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "8CC8D0D119B142D839800BFF71FB71E73AEA7BD4" }, { "b" : "7F3AA8F10000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "89C34D7A182387D76D5CDA1F7718F5D58824DFB3" }, { "b" : "7F3AA8C07000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "DFB85DE42DAFFD09640C8FE377D572DE3E168920" }, { "b" : "7F3AA89F1000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "68220AE2C65D65C1B6AAA12FA6765A6EC2F5F434" }, { "b" : "7F3AA87D4000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "CE17E023542265FC11D9BC8F534BB4F070493D30" }, { "b" : "7F3AA840A000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "B5381A457906D279073822A5CEB24C4BFEF94DDB" }, { "b" : "7F3AA9E53000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "5D7B6259552275A3C17BD4C3FD05F5A6BF40CAA5" }, { "b" : "7F3AA81D7000", "path" : "/usr/lib/x86_64-linux-gnu/libidn.so.11", "elfType" : 3, "buildId" : "E09D3783AD1D0BBCD3204FA01E4EF6D756E18F57" }, { "b" : "7F3AA7FBB000", "path" : "/usr/lib/x86_64-linux-gnu/librtmp.so.1", "elfType" : 3, "buildId" : "8D1CC1204D6B6D33BD1D2C5A2A0516A2234322CF" }, { "b" : "7F3AA7D71000", "path" : "/usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "DB5180B568097E2A4690A5B40D36BD134C893FEE" }, { "b" : "7F3AA7B62000", "path" : "/usr/lib/x86_64-linux-gnu/liblber-2.4.so.2", "elfType" : 3, "buildId" : "5B146086EC173C299BA45F0C92E068D68CC7AD37" }, { "b" : "7F3AA7911000", "path" : "/usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2", "elfType" : 3, "buildId" : "D02A94DB90567C2C45F40C8565809A041FD22A55" }, { "b" : "7F3AA76F7000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "8D9BD4CE26E45EF16075C67D5F5EEAFD8B562832" }, { "b" : "7F3AA73C7000", "path" : "/usr/lib/x86_64-linux-gnu/libgnutls.so.30", "elfType" : 3, "buildId" : "3CE893F6D1382C2C7648DCCB06E71B1C7E0861CC" }, { "b" : "7F3AA7194000", "path" : "/usr/lib/x86_64-linux-gnu/libhogweed.so.4", "elfType" : 3, "buildId" : "B11678F560199547DCF726384EA39153EE0DFABF" }, { "b" : "7F3AA6F5E000", "path" : "/usr/lib/x86_64-linux-gnu/libnettle.so.6", "elfType" : 3, "buildId" : "D6B36C5A463EE0FA84FDD6D5FD3F7726EDB90D54" }, { "b" : "7F3AA6CDE000", "path" : "/usr/lib/x86_64-linux-gnu/libgmp.so.10", "elfType" : 3, "buildId" : "7B3533D5998D20EE1A1BE3F87789B69041E7F620" }, { "b" : "7F3AA6A0C000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5.so.3", "elfType" : 3, "buildId" : "16E3DBC6D048145939BB43BBFD7954D27421B00F" }, { "b" : "7F3AA67DD000", "path" : "/usr/lib/x86_64-linux-gnu/libk5crypto.so.3", "elfType" : 3, "buildId" : "AEB4C08FC47F86C475E9D3996DFE5E9B403ACEBF" }, { "b" : "7F3AA65D9000", "path" : "/lib/x86_64-linux-gnu/libcom_err.so.2", "elfType" : 3, "buildId" : "1E16CB57F699E215A2A8D4EFEF90883BC749B12D" }, { "b" : "7F3AA63CE000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5support.so.0", "elfType" : 3, "buildId" : "DF3219B89E86920E901BAC4A80AA60F2B6134588" }, { "b" : "7F3AA61B3000", "path" : "/usr/lib/x86_64-linux-gnu/libsasl2.so.2", "elfType" : 3, "buildId" : "87783DF8A1058CD150F8886CB36340384093C18F" }, { "b" : "7F3AA5F72000", "path" : "/usr/lib/x86_64-linux-gnu/libgssapi.so.3", "elfType" : 3, "buildId" : "1FE877BE52A424D0636AFD4D35BB330E41D6E0F3" }, { "b" : "7F3AA5D0E000", "path" : "/usr/lib/x86_64-linux-gnu/libp11-kit.so.0", "elfType" : 3, "buildId" : "A0E2D03FF5CF65937F4425D4EFD4D655243809EB" }, { "b" : "7F3AA5AFB000", "path" : "/usr/lib/x86_64-linux-gnu/libtasn1.so.6", "elfType" : 3, "buildId" : "E07E186694852D8F69459C6AB28A53F8DA3CE3B6" }, { "b" : "7F3AA58F7000", "path" : "/lib/x86_64-linux-gnu/libkeyutils.so.1", "elfType" : 3, "buildId" : "3364D4BF2113C4E8D17EF533867ECC99A53413D6" }, { "b" : "7F3AA56EE000", "path" : "/usr/lib/x86_64-linux-gnu/libheimntlm.so.0", "elfType" : 3, "buildId" : "73A8EADBC85860662B24850E71D4AFBE22C33359" }, { "b" : "7F3AA5464000", "path" : "/usr/lib/x86_64-linux-gnu/libkrb5.so.26", "elfType" : 3, "buildId" : "59E742306A4EA2872E061ECCE92F35FADDA75357" }, { "b" : "7F3AA51C2000", "path" : "/usr/lib/x86_64-linux-gnu/libasn1.so.8", "elfType" : 3, "buildId" : "E5C159E415406AE79D21056D752BA949C408B5B1" }, { "b" : "7F3AA4F8F000", "path" : "/usr/lib/x86_64-linux-gnu/libhcrypto.so.4", "elfType" : 3, "buildId" : "7D15576E1F096614D360784E4A01A1F5FAF908C9" }, { "b" : "7F3AA4D79000", "path" : "/usr/lib/x86_64-linux-gnu/libroken.so.18", "elfType" : 3, "buildId" : "481DB33C28D88E43DA6BED65E1A7599407D4D818" }, { "b" : "7F3AA4B71000", "path" : "/usr/lib/x86_64-linux-gnu/libffi.so.6", "elfType" : 3, "buildId" : "9D9C958F1F4894AFEF6AECD90D1C430EA29AC34F" }, { "b" : "7F3AA4948000", "path" : "/usr/lib/x86_64-linux-gnu/libwind.so.0", "elfType" : 3, "buildId" : "57E25072866B2D30CF02EBE7AE623B84F96FA700" }, { "b" : "7F3AA4739000", "path" : "/usr/lib/x86_64-linux-gnu/libheimbase.so.1", "elfType" : 3, "buildId" : "F6F1B4E9F89B716C4A0BA5819BDFFAF4A13EFB91" }, { "b" : "7F3AA44EE000", "path" : "/usr/lib/x86_64-linux-gnu/libhx509.so.5", "elfType" : 3, "buildId" : "C60082E3BB78D0D42868D9B359B89BF66CE5A1A7" }, { "b" : "7F3AA4219000", "path" : "/usr/lib/x86_64-linux-gnu/libsqlite3.so.0", "elfType" : 3, "buildId" : "D9782BA023CAEC26B15D8676E3A5D07B55E121EF" }, { "b" : "7F3AA3FE1000", "path" : "/lib/x86_64-linux-gnu/libcrypt.so.1", "elfType" : 3, "buildId" : "7BDD51353D50310FFA1587E4AA01B40ABE32D582" } ] }}
mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x56187d1b1ef1]
mongod(+0x2472109) [0x56187d1b1109]
mongod(+0x24725ED) [0x56187d1b15ed]
libpthread.so.0(+0x11390) [0x7f3aa87e5390]
libc.so.6(gsignal+0x38) [0x7f3aa843f428]
libc.so.6(abort+0x16A) [0x7f3aa844102a]
mongod(ZN5mongo22invariantFailedWithMsgEPKcRKNSt7_cxx1112basic_stringIcSt11char_traitsIcESaIcEEES1_j+0x0) [0x56187b7843a3]
mongod(_ZN5mongo6sorter12FileIteratorINS_7BSONObjENS_8RecordIdEE4readEPvm+0xD8) [0x56187c7fea78]
mongod(_ZN5mongo6sorter12FileIteratorINS_7BSONObjENS_8RecordIdEE18fillBufferFromDiskEv+0x31) [0x56187c7feae1]
mongod(_ZN5mongo6sorter12FileIteratorINS_7BSONObjENS_8RecordIdEE4moreEv+0x3B) [0x56187c7ff0cb]
mongod(_ZN5mongo6sorter13MergeIteratorINS_7BSONObjENS_8RecordIdENS_27BtreeExternalSortComparisonEE4moreEv+0x3F) [0x56187c7fc91f]
mongod(_ZN5mongo25AbstractIndexAccessMethod10commitBulkEPNS_16OperationContextEPNS_17IndexAccessMethod11BulkBuilderEbbPSt3setINS_8RecordIdESt4lessIS7_ESaIS7_EEPSt6vectorINS_7BSONObjESaISE_EE+0x1E8) [0x56187c7fa718]
mongod(_ZN5mongo19MultiIndexBlockImpl20_dumpInsertsFromBulkEPSt3setINS_8RecordIdESt4lessIS2_ESaIS2_EEPSt6vectorINS_7BSONObjESaIS9_EE+0x1D8) [0x56187c002128]
mongod(_ZN5mongo19MultiIndexBlockImpl19dumpInsertsFromBulkEv+0x24) [0x56187c002424]
mongod(_ZN5mongo19MultiIndexBlockImpl30insertAllDocumentsInCollectionEv+0x8E8) [0x56187c001308]
mongod(+0x118D034) [0x56187becc034]
mongod(ZN5mongo23ErrmsgCommandDeprecated3runEPNS_16OperationContextERKNSt7_cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKNS_7BSONObjERNS_14BSONObjBuilderE+0x46) [0x56187cb38356]
mongod(_ZN5mongo12BasicCommand10Invocation3runEPNS_16OperationContextEPNS_3rpc21ReplyBuilderInterfaceE+0x74) [0x56187cb3b534]
mongod(+0xEB9E34) [0x56187bbf8e34]
mongod(+0xEBC1AE) [0x56187bbfb1ae]
mongod(+0xEBD94C) [0x56187bbfc94c]
mongod(_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE+0x476) [0x56187bbfd926]
mongod(_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE+0x3A) [0x56187bbe9b6a]
mongod(_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE+0xD3) [0x56187bbf5f73]
mongod(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x97) [0x56187bbf1767]
mongod(+0xEB6151) [0x56187bbf5151]
mongod(_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE+0x1A2) [0x56187c95c0f2]
mongod(_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE+0x150) [0x56187bbef840]
mongod(_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE+0xB55) [0x56187bbf2b85]
mongod(_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE+0x35F) [0x56187bbf0d9f]
mongod(_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE+0x11D) [0x56187bbf17ed]
mongod(+0xEB6151) [0x56187bbf5151]
mongod(+0x1C1D665) [0x56187c95c665]
mongod(+0x23C87D4) [0x56187d1077d4]
libpthread.so.0(+0x76BA) [0x7f3aa87db6ba]
libc.so.6(clone+0x6D) [0x7f3aa851141d]
----- END BACKTRACE -----



 Comments   
Comment by Githook User [ 11/Jun/19 ]

Author:

{'name': 'Dianna Hohensee', 'email': 'dianna.hohensee@10gen.com', 'username': 'DiannaHohensee'}

Message: SERVER-17010 each Sorter instance spills to a single file rather than a new file per spill to disk

SERVER-38764 External sorter should use 64-bit integers for file offsets

(cherry picked from commit 48d999c08304b6ede2a9d1f9d9db974b59fe97e2)
Branch: v3.4
https://github.com/mongodb/mongo/commit/c1c761dd865308a15ff75748bf111d0a3ce366d6

Comment by Githook User [ 12/Mar/19 ]

Author:

{'name': 'Dianna Hohensee', 'email': 'dianna.hohensee@10gen.com', 'username': 'DiannaHohensee'}

Message: SERVER-17010 each Sorter instance spills to a single file rather than a new file per spill to disk

(cherry picked from commit 2be7f2677a40a863f336d2964f456c9d87ddc838)

SERVER-38764 External sorter should use 64-bit integers for file offsets

(cherry picked from commit 9dafb7a3e3bafa463ab5951189b670965995dada)
Branch: v3.6
https://github.com/mongodb/mongo/commit/48d999c08304b6ede2a9d1f9d9db974b59fe97e2

Comment by Githook User [ 25/Feb/19 ]

Author:

{'name': 'Louis Williams', 'email': 'louis.williams@mongodb.com', 'username': 'louiswilliams'}

Message: SERVER-38764 External sorter should use 64-bit integers for file offsets

(cherry picked from commit 9dafb7a3e3bafa463ab5951189b670965995dada)
Branch: v4.0
https://github.com/mongodb/mongo/commit/95f1919a6fc3de1193cd0c7c7fb69e7b850edb5f

Comment by Githook User [ 18/Jan/19 ]

Author:

{'username': 'louiswilliams', 'email': 'louis.williams@mongodb.com', 'name': 'Louis Williams'}

Message: SERVER-38764 External sorter should use 64-bit integers for file offsets
Branch: master
https://github.com/mongodb/mongo/commit/9dafb7a3e3bafa463ab5951189b670965995dada

Comment by Louis Williams [ 16/Jan/19 ]

The problem here is the static_cast to an unsigned 32 bit integer truncates the high-order bytes of tellg() when the file is larger than 4GB.

In fact, this entire implementation uses 32-bit unsigned integers when it should be using 64-bit unsigned integers.

Comment by O [ 27/Dec/18 ]

I finished this test and the conclusion is that non-unique indexes and indexes on arrays working on 4.0.4, but not on 4.1.6.

 

Comment by O [ 26/Dec/18 ]

I loaded the same database into a 4.0.4 instance and created an index (unique), and it worked.

I'm testing now with a non unique index, and an index in an array. Soon as I have the results I will post it here.

Comment by O [ 26/Dec/18 ]

I created a dump of this database and imported it. The same bug happens.

Comment by O [ 26/Dec/18 ]

Since when moving a collection from a database to another some indexes are rebuilt, I did it as a test, and I got the following error:

On the shell

> db.runCommand({renameCollection: 'db1.col1', to: 'db2.col1'})

{ "ok" : 0, "errmsg" : "BSONObj size: 1684301409 (0x64646661) is invalid. Size must be between 0 and 16793600(16MB)", "code" : 10334, "codeName" : "BSONObjectTooLarge" }

On server logs

 

2018-12-26T10:13:01.739+0000 I COMMAND [conn79] command db2.tmpw7evh.renameCollection appName: "MongoDB Shell" command: renameCollection { renameCollection: "db1.col1", to: "db2.col1", lsid:

{ id: UUID("fa8f217f-63e7-4138-bf23-03fd55afaf60") }

, $db: "admin" } numYields:0 ok:0 errMsg:"BSONObj size: 1684301409 (0x64646661) is invalid. Size must be between 0 and 16793600(16MB)" errName:BSONObjectTooLarge errCode:10334 reslen:185 locks:{ Global: { acquireCount:

{ r: 3, w: 6, W: 1 }

}, Database: { acquireCount:

{ r: 1, w: 1, W: 1 }

}, Collection: { acquireCount:

{ r: 1, w: 1 }

} } protocol:op_msg 54798794ms

Comment by O [ 24/Dec/18 ]

I finished importing the data into the collection with the indexes already created and the error didn't happen.

Comment by O [ 24/Dec/18 ]

Thank you Kelsey, I will try to repeat the same procedure with the 4.0.4 version sooner as I can.

Now, just for sake of completeness, I'm trying to insert the documents into the database with the indexes (one simple, one unique and one in an array) already created, since I'm more than half way through (about 70%), I will just wait to give you a more complete feedback of the bug.

In case of failure, I will repeat the procedure with 4.0.4 and give you a feedback about it.

Comment by Kelsey Schubert [ 23/Dec/18 ]

Hi ooo,

Thanks for the report, we'll look into it. Please note that that the MongoDB 4.1.x series is a development branch and not suitable for production use. Until MongoDB 4.2 is released containing the work that has gone into the 4.1.x series, we recommend running the latest stable branch, MongoDB 4.0. For more information our on version numbering system see here.

Kind regards,
Kelsey

Comment by O [ 23/Dec/18 ]

The same bug also occurs when creating non-unique indexes.

Generated at Thu Feb 08 04:49:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.