Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-31995

Logged initial sync statistics may exceed 16mb causing fassert

    XMLWordPrintable

    Details

    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Requested:
      v3.6, v3.4
    • Linked BF Score:
      11

      Description

      Hi Team,

      We have a 10 shards (Primary / Secondary / Arbiter) sharded cluster which hosts 70k databases.

      Here's the repartition on the shards:
      (NB: some of our databases are not sharded)

      mongos> db.databases.aggregate({$group:{_id: '$primary', count: {$sum:1}}})
      { "_id" : "clust-users-2-shard10", "count" : 4594 }
      { "_id" : "clust-users-2-shard9", "count" : 8945 }
      { "_id" : "clust-users-2-shard8", "count" : 8624 }
      { "_id" : "clust-users-2-shard1", "count" : 8084 }
      { "_id" : "clust-users-2-shard7", "count" : 4505 }
      { "_id" : "clust-users-2-shard2", "count" : 4769 }
      { "_id" : "clust-users-2-shard6", "count" : 9370 }
      { "_id" : "clust-users-2-shard4", "count" : 4717 }
      { "_id" : "clust-users-2-shard3", "count" : 10217 }
      { "_id" : "clust-users-2-shard5", "count" : 5953 }
      

      We're currently experiencing issues to resync this shard from scratch with the following error:

      2017-11-16T05:49:33.245+0100 I -        [replication-115] Assertion: 10334:BSONObj size: 32985739 (0x1F7528B) is invalid. Size must be between 0 and 16793600(16MB) First element: databasesCloned: 10191 src/mongo/bson/bsonobj.cpp 58
      

      On another cluster with the same architecture but less databases per shards, we do not encounter this issue.

      We plan to upgrade from version 3.4.4 to 3.4.10 but we haven't found anything related to this issue in changelog.
      Is this a known issue or do you have more information about this?

      Thanks.

      Regards,
      Benoit

        Attachments

        1. clust-users-2-shard3-2.log
          15 kB
        2. mongod-logs.txt
          9 kB

          Issue Links

            Activity

              People

              Assignee:
              benety.goh Benety Goh
              Reporter:
              BenoitSIB Benoit Bui [X]
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: