Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-13806

Need better detection and reporting of the existence of jumbo chunks

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Won't Fix
    • None
    • None
    • Sharding
    • None
    • Sharding
    • ALL
    • Hide 

      sh.enableSharding("test");
      		 
      sh.shardCollection("test.foo",{a:1});
      		 
      sh.splitAt("test.foo",{a:1000});
      		 
      for (var i=0; i<10000000; i++) {db.getSiblingDB("test").foo.insert({a:1, padding:"xxxxxxxx...xxxxx"})}

      Run the above commands and observe that chunk MinKey -> 1000 is not marked as jumbo in the chunk meta data, even after it has grown past the max chunk size for the cluster.

      Show
      sh.enableSharding("test"); sh.shardCollection("test.foo",{a:1}); sh.splitAt("test.foo",{a:1000}); for (var i=0; i<10000000; i++) {db.getSiblingDB("test").foo.insert({a:1, padding:"xxxxxxxx...xxxxx"})} Run the above commands and observe that chunk MinKey -> 1000 is not marked as jumbo in the chunk meta data, even after it has grown past the max chunk size for the cluster.

    Description

      As far as I can see, a jumbo chunk is only marked as such when the balancer has attempted to migrate it to another shard (Chunk::markAsJumbo is only called from Balancer::_moveChunks).

      This means that there's no indication of jumbo chunks unless:

      • chunk imbalance has grown past the threshold at which the balancer tries to move chunks, and
      • the balancer has attempted to move a jumbo chunk to redress the imbalance

      It would be good to have more comprehensive and early warning of jumbo chunks. This checking could perhaps be added to the splitVector handling in mongod.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. SERVER-21931 Only mark chunks as jumbo if split fails because the chunk is too large

          • Major - P3 - Major loss of function.
          • Closed
          related to

          Bug - A problem which impairs or prevents the functions of the product. SERVER-44088 Autosplitter seems to ignore some fat chunks

          • Major - P3 - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. SERVER-13024 Clear the 'jumbo' flag from the chunk document when the chunk is no longer over-sized

          • Major - P3 - Major loss of function.
          • Backlog

          Activity

            People

              backlog-server-sharding Backlog - Sharding Team
              jon.rangel@mongodb.com Jon Rangel (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: