Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-5452

Assertion: 10334:Invalid BSONObj error when doing Map/Reduce

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Works as Designed
    • Affects Version/s: 2.0.3
    • Fix Version/s: None
    • Component/s: MapReduce
    • Labels:
      None
    • Environment:
      RHEL 5.8
    • Operating System:
      Linux

      Description

      Hello all,

      I have been using Mongo in non-sharded mode for quite sometime and have just recently tried my hand at sharding data. To do so I have 3 machines setup as mongod with 1 of them also serving as my mongos and config server.

      I have ~212 thousand documents in the sharded collection but each document has what can be a pretty long array.

      This dataset was working fine for about a day but after doing an update where I added a new field to most of the documents I am getting the following error:

      Thu Mar 29 14:56:22 [conn5166] Assertion: 10334:Invalid BSONObj size: 18303487 (0xFF491701) first element: 0:

      { count: 6294467.0 }

      The complete log from mongos is here:

      Thu Mar 29 14:56:22 [conn5166] Assertion: 10334:Invalid BSONObj size: 18303487 (0xFF491701) first element: 0: { count: 6294467.0 }
      0x584432 0x508461 0x9bacbd 0x9bb0c8 0x9b3e87 0x9b43a5 0x9ca9bb 0x97cc14 0x97e20f 0x940e25 0x9441b1 0x8869d7 0x88df49 0xaa37d6 0x637497 0x3d32e0677d 0x3d31ed49ad 
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x112) [0x584432]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x471) [0x508461]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo2mr9JSReducer7_reduceERKSt6vectorINS_7BSONObjESaIS3_EERS3_Ri+0xf2d) [0x9bacbd]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo2mr9JSReducer6reduceERKSt6vectorINS_7BSONObjESaIS3_EE+0x88) [0x9bb0c8]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo2mr5State14reduceInMemoryEv+0xe7) [0x9b3e87]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo2mr5State9checkSizeEv+0xf5) [0x9b43a5]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo2mr16MapReduceCommand3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0xa1b) [0x9ca9bb]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x6a4) [0x97cc14]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x6ff) [0x97e20f]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x35) [0x940e25]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x11e1) [0x9441b1]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod [0x8869d7]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x559) [0x88df49]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x76) [0xaa37d6]
       /export/mongodb-linux-x86_64-2.0.3/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x287) [0x637497]
       /lib64/libpthread.so.0 [0x3d32e0677d]
       /lib64/libc.so.6(clone+0x6d) [0x3d31ed49ad]
      Thu Mar 29 14:56:22 [conn5166] warning: ClientCursor::YieldLock not closed properly
      Thu Mar 29 14:56:22 [conn5166] mr failed, removing collection :: caused by :: 10334 Invalid BSONObj size: 18303487 (0xFF491701) first element: 0: { count: 6294467.0 }
      Thu Mar 29 14:56:22 [conn5166] CMD: drop lgt8.tmp.mr.bwa_mapping_49
      Thu Mar 29 14:56:22 [conn5166] CMD: drop lgt8.tmp.mr.bwa_mapping_49_inc
      Thu Mar 29 14:56:22 [conn5166] command lgt8.$cmd command: { mapreduce: "bwa_mapping", map: "    function() {
              var conds = [];
              var thiselm = this;
              this.hits.forEach(
                  function(h) {
                      var good = t...", reduce: "    function(key,values) {
              var result = {count:0.0};
              values.forEach(function(value) {
                  result.count += value.count;
             ...", query: {}, out: "tmp.mrs.bwa_mapping_1333047154_116" } ntoreturn:1 exception: Invalid BSONObj size: 18303487 (0xFF491701) first element: 0: { count: 6294467.0 } code:10334 reslen:187 228442ms

      I have tried running repairDatabase(), which ran successfully but does not take care of the problem. I have also run validate() on the collection and validate() reports everything as OK (even when run in verbose mode). I am able to query this database without error however I am unable to perform the map reduce described above (which is counting a subset of the array elements and doing a group by). I am able to perform a map reduce on the elements of the document NOT contained in the array.

      I am not sure how much of this information is helpful but I thought I would include it just in case.

      Any help on this would be greatly appreciated.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              antoine Antoine Girbal
              Reporter:
              driley David Riley
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: